EP1730889B1 - Methods and devices for high network availability - Google Patents

Images

Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L1/00—Arrangements for detecting or preventing errors in the information received
  • H04L1/22—Arrangements for detecting or preventing errors in the information received using redundant apparatus to increase reliability
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L1/00—Arrangements for detecting or preventing errors in the information received
  • H04L1/24—Testing correct operation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L43/00—Arrangements for monitoring or testing data switching networks
  • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
  • H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
  • H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L45/00—Routing or path finding of packets in data switching networks
  • H04L45/22—Alternate routing
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L45/00—Routing or path finding of packets in data switching networks
  • H04L45/28—Routing or path finding of packets in data switching networks using route fault recovery
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
  • H04L41/06—Management of faults, events, alarms or notifications

Definitions

• the present invention relates to high availability in networks. More specifically, the present invention relates to methods and devices for detecting the failure of an active path or paths and switching over to a standby or remaining active path.
• the determination of when to switch over to a redundant path is made by a device in the first location, which may be a host, a storage device, etc. This determination is based upon sets of timers that are used to expire a potentially failed transaction on the path. For example, "keepalive” or similar messages are often exchanged between devices in the first and second locations to discover if the end service and network transport is available. If a "keepalive” message is not received within a timer period, a switchover to the redundant path is triggered. The "keepalive” mechanism is required due to the fact that no direct linkage exists between the end device and the network transport to signal a broken path.
• the end devices it is the responsibility of the end devices to detect the failed network path using mechanisms such as "keepalive.”
• the timer periods are quite long (e.g., 60 to 180 seconds), because it is undesirable to switch unnecessarily to the redundant path or experience any flapping of the service between two feasible paths.
• US 6662308, Baroni et al describes a network architecture comprised of nodes and a transport network capable of recovering from failures within the network.
• the network architecture incorporates select functions into the nodes for managing the data flow between individual edge routers and their connection through core routers to the transport network allowing an individual node to recover independently without signalling other nodes over the transport network.
• Ethernet protection system that includes an Ethernet communication device operable to be connected to first and second Ethernet lines forming a parallel connection.
• the Ethernet communication device transmits and receives data over the first Ethernet line, and upon detecting a failure in the first Ethernet line, automatically selects the second Ethernet line to transmit and receive data.
• EP1158736 Kiuchi et al , detects network faults or congestion of traffic at an interface of a gateway system for connecting an IP network and a telephone network on the side of the IP network and to allow the IP network to control alternative routing at the telephone network. Network faults and congestion of traffic on the side of the IP network are monitored by the gateway system having an interface function between the telephone network and the IP network.
• the network device is a gateway of a first network and the end device is a host or a storage device on the first network.
• the network device includes a linked interface, the administrative state of which is determined by the operational state of one or more tracked interfaces.
• the tracked interfaces may be in the same network device or other network devices along an active path between two or more end devices, and may be physical or virtual.
• the linked interface will bring down a link between the network device and the end device, thereby causing a switchover to a redundant path.
• Other implementations involve direct notification of the network device that the link has failed or is about to fail.
• Some implementations of the invention provide a method of maintaining high availability in a network.
• the method includes the step of deriving a state of a linked interface of a first network device from the operational state of a first tracked interface, wherein the linked interface and the first tracked interface from part of a first data path between a first end device and a second end device.
• the method also includes the step of notifying the first end device of a failure in the first data path, for example by bringing down the first data pathway.
• the method may include the step of switching to a second data pathway.
• the method may also include the step of deriving the state of the linked interface from the operational stat of at least a second tracked interface.
• the state of the linked interface may be determined by the use of a weighting function applied to at least the first tracked interface and the second tracked interface.
• the deriving step may involve applying a damping function to a recovery of the linked interface.
• Some embodiments of the invention provide a network device that includes a first tracked interface having an operational state and a linked port configured to derive a linked port state from the operational state of the tracked interface.
• the linked port and the first tracked interface form part of a first data path between a first end device and a second end device.
• the linked port may be further configured to notify the first end device when the linked port state is a down state.
• the first tracked interface may be a physical interface or a virtual interface.
• the first tracked interface may be one of a Fibre Channel interface or an Ethernet interface.
• the first tracked interface may be part of the network device or part of another network device.
• the linked port may be a Fibre Channel port.
• the linked port may be further configured to drop a connection with the first end device when the linked port state is a down state.
• the linked port state may be determined by applying a weighting function to the first tracked interface and a second tracked interface.
• the linked port state may be determined by applying a damping function to an operational state.
• the network device may also include a routing table, wherein the linked port state is based upon an update in the routing table.
• Alternative implementations of the invention provide a method of maintaining high availability in a network.
• the method includes the following steps: receiving a signal indicating a condition of a first data path between a first end device and a second end device; deriving a state of a linked port of a first network device from the signal, the linked port forming part of the first data path; and notifying the first end device of the state.
• the condition of the receiving step may indicate that the first data path is down.
• the condition of the receiving step may indicate that the first data path will be down at a future time.
• the notifying step may involve dropping a link between the first network device and the first end device.
• Alternative embodiments of the invention provide a computer program embodied in a machine-readable medium.
• the computer program includes instructions for controlling at least one device to perform the following steps: deriving a state of a linked interface of a first network device from the operational state of a first tracked interface, the linked interface and the first tracked interface forming part of a first data path between a first end device and a second end device; and notifying the first end device of the state of the linked interface.
• An aspect of the invention is set out in claim 1. Another aspect of the invention is set out in claim 9.
• the techniques of the present invention will mainly be described in the context of communications between two or more storage area networks that use the Fibre Channel protocol.
• the techniques of the present invention can be applied to a variety of different protocols and networks.
• the solutions afforded by the invention are equally applicable to non-Fibre Channel networks.
• numerous specific details are set forth in order to provide a thorough understanding of the present invention. The present invention may be practiced without some or all of these specific details. In other instances, well known process operations have not been described in detail in order not to obscure the present invention.
• Fig. 1 is a simplified network diagram that will be used to describe some implementations of the present invention.
• Client device 105 of Fig. 1 communicates with client device 125 via an intervening network 115 (here, Internet 115).
• Client device 105 may be a host, a storage device, or a similar device.
• Client device 125 is, or at least includes, a storage device.
• client devices 105 and 125 may be, for example, devices within storage area networks in separate geographic locations.
• Data may be transmitted between devices 105 and 125 according to the Small Computer System Interface (“SCSI”) standard or by other protocols.
• SCSI Small Computer System Interface
• data may be transferred according to the Fiber Connection (“FICON”) standard or the Enterprise System Connection (“ESCON”) standard.
• FICON Fiber Connection
• ESCON Enterprise System Connection
• Network 115 may be any type of network suitable for efficient communication between devices 105 and 125, such as a metro optical transport or the Internet. Accordingly, a variety of protocols may be used for communication on network 115, such as Internet Protocol ("IP”), Fibre Channel (“FC”), FC over IP (“FCIP”), Internet SCSI (“iSCSI,” an IP-based standard for linking data storage devices over a network and transferring data by carrying SCSI commands over IP networks) or Dense Wavelength Division Multiplexing (“DWDM,” an optical technology used to increase bandwidth over existing fiber optic backbones).
• IP Internet Protocol
• FC Fibre Channel
• FCIP FC over IP
• iSCSI Internet SCSI
• iSCSI an IP-based standard for linking data storage devices over a network and transferring data by carrying SCSI commands over IP networks
• DWDM Dense Wavelength Division Multiplexing
• Gateway 110 is a network device (e.g., a switch) disposed between device 105 and network 115.
• gateway 120 is a network device disposed between device 125 and network 115.
• path 140 is a primary data path from device 105 to device 125, via network 115.
• link 130 is formed between device 105 and port 132 of gateway 110.
• link 135 is formed between port 135 and network 115.
• Network devices 160, 165, 170 and 175 form part of path 140.
• network devices 160, 165, 170 and 175 are merely illustrative of a number of network devices along path 140 and within network 115. If some part of path 140 fails, device 105 causes a switchover and data are transmitted between devices 105 and 125 via redundant path 150.
• a network device such as gateway 110 and an end device such as device 105.
• the network device includes a linked interface (such as port 132), the administrative state of which is determined by the operational state of one or more tracked interfaces.
• the tracked interfaces may be in the same network device (e.g., port 136) or other network devices, and may be physical or virtual.
• the linked interface will bring down a link between the network device and the end device (e.g., link 130), thereby causing a switchover to a redundant path (e.g., path 150).
• a damping function (or similar algorithm) may be applied, such that a tracked interface must remain in a "down" state for a predetermined period of time before a linked interface reaches a "down" administrative state and brings the link down.
• a weighting function or metric may be applied to determine the administrative state of the linked interface when one such path fails.
• Fig. 2 illustrates a simplified yet more realistic setting for devices 105 and 125, according to some embodiments of the invention.
• Client SAN 200 includes network devices 235, 240 and 245, hosts 205 and 210, and storage devices 220 and 225.
• network devices 235, 240 and 245 hosts 205 and 210
• storage devices 220 and 225 storage devices 220 and 225.
• device 105 of Fig. 1 is a host device such as host 205 of Fig. 2 .
• the linked interface will be in switch 245 and will bring down link 206 when the tracked interface is in a "down" operational state.
• device 105 of Fig. 1 may be a storage device such as storage device 225.
• the linked interface will be in switch 245 and will bring down link 226 when the linked interface is in a "down" administrative state.
• Fig. 3 is a flow chart that illustrates method 300 according to some implementations of the present invention.
• a first data path is established between client devices in different geographic locations, e.g., path 140 between devices 105 and 125 of Fig. 1 .
• the data path includes one or more linked interfaces and at least one tracked interface, which may be physical or virtual.
• step 310 it is determined whether a link on the first data path has failed (or will fail at some known time in the future). In some implementations, the determination is made directly, as in when the tracked interface is in the same network device as the linked interface. Alternative implementations, e.g., wherein the tracked interface is not in the same network device as the linked interface, will be discussed below.
• the determination of step 310 may be made by direct communication of state information from another device, e.g., via information in a packet or frame header.
• a direct communication is also referred to as an "external indication.”
• the external indication may indicate, for example, that a portion of the first data path has gone down or will be going down at some time in the future. For example, if a network administrator plans to perform an upgrade or maintenance on a device on data path 140, he or she could send a direct communication from a host device to a device having a linked interface (e.g., gateway 110), indicating that the device will be taken off-line at a particular time.
• a linked interface e.g., gateway 110
• an acknowledged messaging protocol (unicast, multicast, or broadcast) is used to signal neighboring network devices or a group of end devices that an element of an active network path is about to be taken down, thereby enacting the failover mechanism without requiring each end device to cycle through a "keepalive" timeout duration.
• the message uses the standard TCP/IP frame format and has a payload that includes several pieces of information, such as: (1) the identity of the device sending the alert; (2) a message identifier indicating the nature of the message (i.e., going-down-now, going-down-in-future, protocol initialization, keepalive, etc); (3) a duration until the device is downed; (4) a device group ID; and/or (5) a protocol version ID.
• step 310 is made by reference to the operational state of one or more tracked interfaces. If no link on the first data path has failed, the administrative state of the linked interface remains "up” (step 315) and the first data path is maintained (step 320).
• the administrative and/or operational state of the linked interface is set to "down" (step 330).
• a damping function is applied, wherein the administrative and/or operational state of the linked interface is not immediately set to "down” if a link on the first data path has failed.
• a weighting or similar function may be applied to determine whether the administrative and/or operational state of the linked interface is set to "down” in step 330.
• a link to an end device (e.g., link 130 to device 105) is brought down. This causes the end device to switch over to a redundant second data path, such as path 150 of Fig. 1 .
• a redundant second data path such as path 150 of Fig. 1 .
• gateways 110 and 120 can detect a failure in path 140 in less time than a "keeaplive" timer period, the switchover will occur in less time than would be the case with prior art methods. This is true regardless of the protocol used to transfer data, the type of client device involved or the algorithm(s) used by, e.g., device 105 or device 125 to determine when path 140 would otherwise be brought down.
• Fig. 4 is a flowchart that will be used to describe in more detail some of the interface-tracking implementations of the present invention.
• a first data path is established between client devices in different geographic locations, e.g., path 140 of Fig. 1 .
• the data path includes a linked interface and at least one tracked interface, which may be physical or virtual.
• the linked interface could be port 132 of gateway 110, as depicted in Fig. 1 .
• the linked interface could be within another network device within the fabric of a SAN, such as network device 235 or 240 shown in Fig. 2 .
• the tracked interface(s) may be in the same network device (e.g., port 136) or in another network device.
• the linked interface is in network device 235 shown in Fig. 2
• the tracked interface(s) could be in network device 245.
• step 410 it is determined whether a tracked interface (physical or logical) on the first data path has failed. According to method 400, the determination of step 410 is made by reference to the operational state of one or more tracked interfaces. If no link on the first data path has failed, the administrative state of the linked interface remains "up” (step 415) and the first data path is maintained (step 420).
• the determination of step 410 may be made indirectly.
• a determination that an interface has failed may be made by the communication of routing table information from the network device with the tracked interface to the network device with the linked interface. For example, there is often dynamic updating of routes in routing tables of network devices in a path. If the routing tables in the network devices of path 140 were dynamically updated, information regarding the failure of any of the links on path 140 would soon be reflected in all devices in the path. If, e.g., network device 170 failed, this information would pass via routing table updates to a routing table of network device 165, then to a routing table of network device 160 and then to a routing table of gateway 110. Such updates are typically made every 3 to 5 seconds.
• Some network devices may be configured to support a novel frame format, known as Extended Inter-switch Link (“EISL”) format, which is the subject of other pending patent applications assigned to Andiamo Systems.
• EISL Extended Inter-switch Link
• the EISL format allows a single network device to process frames or packets having different formats.
• a network device configured to support EISL may process both FC frames and Ethernet frames.
• the EISL format also supports VLANs, VSANs and similar features.
• An EISL format allows the implementation of a fibre channel network with features and functionality beyond that provided by Inter-Switch Link (“ISL") format.
• the EISL format allows a port (known herein as a "trunking port") to transport frames of more than one format.
• a trunking port can switch Ethernet and Fiber Channel (“FC”) frames and is adaptable to transmitting frames of other formats as they are developed.
• An EISL header is used on EISL links to enable this transportation of different frame types.
• the EISL format allows the implementation of multiple virtual storage area networks (VSANs) on a single physical network.
• VSANs virtual storage area networks
• a tracked interface may be a logical interface such as virtual SAN ("VSAN") number of a trunking port.
• VSAN virtual SAN
• client network 505 includes hosts 510 and 511 and gateways 512 and 513.
• Client network 525 includes gateways 520 and 521, and storage devices 526 and 527.
• Network 515 connects client networks 505 and 525.
• Host 510 communicates with storage device 526 and storage device 527 via VSAN1.
• Host 511 communicates with storage device 526 and storage device 527 via VSAN2.
• port 556 of gateway 513 and port 523 of gateway 521 are TE ports capable of carrying traffic of both VSAN1 and VSAN2.
• Port 536 of gateway 512 and port 522 of gateway 520 could be TE ports or an E port, because they only carrying traffic for VSAN1.
• tracked interfaces for VSAN2 could include ports 523 and 556.
• the state of the logical interfaces is tracked: if VSAN2 were turned off (e.g., by a network administrator), the administrative and/ or operational state of linked interface 554 would become "down" (step 430) and physical link 590 would be dropped (step 435). This would be the case even if all physical links corresponding to the tracked logical interfaces were functioning. Bringing down link 590 could trigger host 511 to switch over to a redundant data path (not shown) in step 440.
• a damping function may be applied after determining that a tracked interface is down. This may be desirable to avoid bringing down a data path and causing a switchover when a tracked interface is down for only a short period of time.
• a tracked interface goes down 3 times within a period T.
• the state of the tracked interface is "up.”
• the administrative state of the linked interface is set to that of the tracked interface at the end of a predetermined period T.
• the administrative state of the linked interface would be set to "up” in the example shown in Fig. 6 .
• the administrative state of the linked interface could be set according to the amount of time during period T that the tracked interface is up or down, the number of times that the tracked interface is down, etc.
• Fig. 7 depicts a situation in which first data path 740, which links device 705 in a first location and device 725 in a second location, includes parallel data paths having different data transmission rates.
• link 730 is capable of delivering data to gateway 710 at 1 Gbps.
• These data may be transmitted from gateway 710 to network 715 via three different pathways: path 737 has a transmission rate of 400 Mbps, path 742 has a transmission rate of 600 Mbps and path 747 has a transmission rate of 50 Mbps. Because path 747 has a relatively low data transmission rate, data path 740 could still provide acceptable transmission rates even if path 747 failed.
• a "metric" or weighting function is assigned to indicate the relative importance of links 737, 742 and 747.
• link 737 is assigned a metric of 50
• link 741 is assigned a metric of 100
• link 747 is assigned a metric of 10.
• These metrics may be regarded as being assigned to corresponding ports 736, 741 and 746 of gateway 710.
• Fig. 8 illustrates the application of these metrics to method 800 of the present invention.
• a first data path e.g., path 740 of Fig. 7
• step 810 it is determined whether a tracked interface is down. If so, it is then determined whether the aggregate metric of the tracked interfaces (e.g., ports 736, 741 and 746 of gateway 710) falls below a certain threshold (step 825).
• the administrative state of linked port 732 would be "down" (step 830) and link 730 would be dropped (step 835) if it were determined in step 825 that the total metric of links 737, 742 and 747 had dropped below a certain number, e.g., 100. If either link 737 or link 747 were to fail, path 740 would not be dropped. However, if link 742 were to fail, path 740 would be dropped and device 705 would switch over to redundant pathway 750 (step 840).
• Fig. 9 illustrates an example of a network device that may be configured to implement some methods of the present invention.
• Network device 960 includes a master central processing unit (CPU) 962, interfaces 968, and a bus 967 (e.g., a PCI bus).
• interfaces 968 include ports 969 appropriate for communication with the appropriate media.
• one or more of interfaces 968 includes at least one independent processor 974 and, in some instances, volatile RAM.
• Independent processors 974 may be, for example ASICs or any other appropriate processors. According to some such embodiments, these independent processors 974 perform at least some of the functions of the logic described herein.
• one or more of interfaces 968 control such communications-intensive tasks as media control and management. By providing separate processors for the communications-intensive tasks, interfaces 968 allow the master microprocessor 962 efficiently to perform other functions such as routing computations, network diagnostics, security functions, etc.
• the interfaces 968 are typically provided as interface cards (sometimes referred to as "linecards"). Generally, interfaces 968 control the sending and receiving of data packets over the network and sometimes support other peripherals used with the network device 960. Among the interfaces that may be provided are FC interfaces, Ethernet interfaces, frame relay interfaces, cable interfaces, DSL interfaces, token ring interfaces, and the like. In addition, various very high-speed interfaces may be provided, such as fast Ethernet interfaces, Gigabit Ethernet interfaces, ATM interfaces, HSSI interfaces, POS interfaces, FDDI interfaces, ASI interfaces, DHEI interfaces and the like.
• CPU 962 may be responsible for implementing specific functions associated with the functions of a desired network device. According to some embodiments, CPU 962 accomplishes all these functions under the control of software including an operating system (e.g., Cisco IOS, a proprietary operating system developed by Cisco Systems, Inc., etc.) and any appropriate applications software.
• an operating system e.g., Cisco IOS, a proprietary operating system developed by Cisco Systems, Inc., etc.
• CPU 962 may include one or more processors 963 such as a processor from the Motorola family of microprocessors or the MIPS family of microprocessors. In an alternative embodiment, processor 963 is specially designed hardware for controlling the operations of network device 960. In a specific embodiment, a memory 961 (such as non-volatile RAM and/or ROM) also forms part of CPU 962. However, there are many different ways in which memory could be coupled to the system. Memory block 961 may be used for a variety of purposes such as, for example, caching and/or storing data, programming instructions, etc.
• network device may employ one or more memories or memory modules (such as, for example, memory block 965) configured to store data, program instructions for the general-purpose network operations and/or other information relating to the functionality of the techniques described herein.
• the program instructions may control the operation of an operating system and/or one or more applications, for example.
• the present invention relates to machine-readable media that include program instructions, state information, etc. for performing various operations described herein.
• machine-readable media include, but are not limited to, magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROM disks; magneto-optical media; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory devices (ROM) and random access memory (RAM).
• ROM read-only memory devices
• RAM random access memory
• the invention may also be embodied in a carrier wave traveling over an appropriate medium such as airwaves, optical lines, electric lines, etc.
• program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter.
• Fig. 9 illustrates one specific network device of the present invention, it is by no means the only network device architecture on which the present invention can be implemented. For example, an architecture having a single processor that handles communications as well as routing computations, etc. is often used. Further, other types of interfaces and media could also be used with the network device.
• the communication path between interfaces/linecards may be bus based (as shown in Fig. 9 ) or switch fabric based (such as a cross-bar).

Landscapes

• Engineering & Computer Science (AREA)
• Computer Networks & Wireless Communication (AREA)
• Signal Processing (AREA)
• Environmental & Geological Engineering (AREA)
• Data Exchanges In Wide-Area Networks (AREA)

Description

Claims (15)

Applications Claiming Priority (2)

Publications (3)

Family

ID=34912372

Family Applications (1)

Country Status (4)

Families Citing this family (61)

Family Cites Families (20)

• 2004
  • 2004-03-04 US US10/794,880 patent/US7428214B2/en not_active Expired - Fee Related
• 2005
  • 2005-02-23 CN CNB2005800068681A patent/CN100499519C/en active Active
  • 2005-02-23 WO PCT/US2005/006091 patent/WO2005094004A1/en active Application Filing
  • 2005-02-23 EP EP05723798.4A patent/EP1730889B1/en active Active

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events