¿ªÆô×ó²à

»ùÓÚIBAµÄSRP TargetµÄÑо¿ÓëʵÏÖ

Õ¹»á

ÒýÑÔ
InfiniBandÊÇÒ»ÖÖÐµĻ¥Á¬¼¼Êõ£¬Ëü½«I£¯OϵͳÓëCPU£¯Mem·Ö¿ª¡£¿Ë·þÁ˵±Ç°I£¯O½á¹¹µÄÖÖÖÖ±×¶Ë£¬ÌṩÁ˸ߴø¿í¡¢µÍÑÓ³Ù¡¢¿ÉÀ©Õ¹µÄI£¯0»¥Á¬¡£SRPЭÒéÊÇInfini-BandÖеÄÒ»ÖÖͨÐÅЭÒ飬ÔÚInfiniBandÖн«SCSIÃüÁî½øÐдò°ü£¬ÔÊÐíSCSIÃüÁîͨ¹ýRDMA(Ô¶³ÌÖ±½ÓÄÚ´æ·ÃÎÊ)ÔÚ²»Í¬µÄϵͳ֮¼ä½øÐÐͨÐÅ£®ÊµÏÖ´æ´¢É豸¹²ÏíºÍRDMAͨÐÅ·þÎñ¡£



1 SRP Target¼ò½é
SRP Target½«ÏµÍ³ÖеĴ洢É豸ÐÅÏ¢Ó³Éäµ½IBÍøÂçÉÏ£®ÈÃSRP Initiator¶ËµÄ·þÎñÆ÷Äܹ»ÔÚIBÍøÂçÉÏ»ñÈ¡µ½ÕâЩSCSI´æ´¢É豸µÄÐÅÏ¢£¬²¢¿Éͨ¹ýÓëSRP Target¶Ô´æ´¢É豸µÄ½øÐзÃÎÊ¡£


10-01-15-46-1.jpg


ͼ1ΪSRP TargetµÄЭÒéÕ»½á¹¹£¬ÈçͼËùʾ£¬ÓëIBÁ´Â·Ö±½ÓÁ¬½ÓµÄÊÇInfiniBand HCA£¬ËüÓëSRP Initiator¶ËµÄHCA½¨Á¢ÁËRDMAͨµÀ£¬¸ºÔðÌṩRDMAͨÐÅ·þÎñ¡£SRP TargetÇøÓò°üÀ¨ÁËSCSI Target¡¢SRP Target DriveºÍInfiniBand DriverÈý¸ö²¿·Ö¡£



2 SRP TargetµÄʵÏÖ
2£®1 ³õʼ»¯SRP TargetÄ£¿é
SCSI TargetµÄ³õʼ»¯ÊÇÖ¸»ñȡϵͳÖеÄSCSI´æ´¢É豸ÐÅÏ¢²¢×¢²á£¬ÒÔʹģ¿é¾ßÓд¦ÀíSCSIÃüÁîµÄ¹¦ÄÜ¡£SRPTargetµÄ³õʼ»¯¹Ø¼üÊÇʹSRP TargetÓëSCSI Target½¨Á¢ÁªÏµ£¬Ê¹Æä¾ßÓд¦ÀíSRPÐÅÏ¢¡¢½øÐÐSCSIÃüÁîµÄת»»¡¢½øÐÐm¹ÜÀíÒÔ¼°ÌṩRDMAͨµÀ·þÎñµÈ¹¦ÄÜ¡£³õʼ»¯Íê³ÉÖ®ºó£¬SRP Initiator±ã¿ÉÒÔ»ñÈ¡IBÍøÂçÉϵÄSCSI´æ´¢É豸ÐÅÏ¢¡£
2£®2 ¶Á²Ù×÷µÄʵÏÖ
¶Á²Ù×÷ÊÇÖ¸SRP Initiator·þÎñÆ÷´ÓSRP Target·þÎñÆ÷¶ÁÈ¡ÐÅÏ¢µÄ¹ý³Ì¡£¶Á²Ù×÷°üÀ¨ÁËSRP Initiator´ÓSRP Target·þÎñÆ÷»ñÈ¡SCSI´æ´¢É豸ÐÅÏ¢¡¢²é¿´´æ´¢É豸ÈÝÁ¿´óС£¬¶ÁÈ¡´æ´¢É豸ÎļþÐÅÏ¢µÈ²Ù×÷¡£ÊµÏÖ¾ßÌå¹ý³ÌÈçͼ2Ëùʾ¡£


10-01-15-55-1.jpg


µ±Íê³ÉÊý¾Ý´«ÊäÒÔºó¡£SRP Target¾ÍÒª¶ÔÖ´Ðйý³ÌÖÐËùÕ¼ÓеÄ×ÊÔ´½øÐÐÊÍ·Å£¬ÎªÏÂÒ»¸öÈÎÎñÌÚ³ö×ÊÔ´£¬SRP TarÒ»get¾Í´¦Óڵȴý½ÓÊÕÏÂÒ»¸öÈÎÎñµÄ״̬ÖС£
2£®3 д²Ù×÷ʵÏÖ¹ý³Ì
д²Ù×÷ÊÇÖ¸SRP Initiator·þÎñÆ÷´ÓSRP Target·þÎñÆ÷дÐÅÏ¢µÄ¹ý³Ì¡£Ð´²Ù×÷°üÀ¨ÁËSRP Initiator·þÎñÆ÷ÏòSRPTarget·þÎñÆ÷ÉϵĴ洢É豸дÈëÊý¾Ý£¬¶Ô´æ´¢É豸½øÐиñʽ»¯µÈ²Ù×÷¡£


10-01-15-41-1.jpg


ÊÍ·ÅSCSIÃüÁîÕ¼ÓÐ×ÊÔ´µÄ²Ù×÷¹ý³ÌÓëǰÊö´¦Àí¶ÁÃüÁîµÄ¹ý³ÌÊÇÒ»Öµġ£

ÊÕ²Ø
ËÍÔÞ
·ÖÏí

·¢±í»Ø¸´

µÇ¼ºó²ÅÄÜÆÀÂÛ

ÆÀÂÛÁÐ±í£¨1£©

5350210 Â¥Ö÷

2£®4 ´¦ÀíÒì³£ÖжÏ
µ±SRP Initiatorͨ¹ýRDMAͨµÀÏòSRP Target·¢ËÍSRPÇëÇóÐÅÏ¢£®ÒªÇóRDMAͨµÀ¶ÔÓ¦µÄSRP Target¶Ë¿Ú½ÓÊÕSRPÇëÇóÐÅÏ¢¡£¶Ë¿ÚÖ´ÐнÓÊÕ²Ù×÷£¬µ«ÊdzöÏÖÒì³£Öжϣ¬½ÓÊÕSRPÇëÇóÐÅϢʧ°Ü£¬¾ÍÒª½øÐÐÏàÓ¦µÄ´¦Àí´ëÊ©£®¾ßÌå²½ÖèÈçÏ£º


(1)SRP TargetµÄ¶Ë¿Ú¼ì²âRDMAͨµÀÖÐÓÐSRPÇëÇóÐÅÏ¢£¬È·ÈÏΪ½ÓÊÕÍê³É²Ù×÷ʧ°Ü£¬Î´½ÓÊÕµ½SRPÇëÇóÐÅÏ¢£¬Á¢¼´½øÐвÙ×÷ʧ°ÜϵĴ¦Àí·½Ê½¡£
(2)ÊÍ·ÅRDMA»º³åÇøÏà¹Ø×ÊÔ´£¬ÖØÐ³õʼ»¯Ïà¹ØÊôÐÔÐÅÏ¢¡£
2£®5 Ð¶ÔØSRPTarget
µ±½øÐÐÐ¶ÔØSRP Targetʱ£¬Ö÷Òª½øÐÐÊÍ·ÅSRP TargetÄ£¿éºÍSCSIÄ£¿éËùÕ¼ÓÐ×ÊÔ´¡£¾ßÌåµÄ²½ÖèÈçÏ£º
(1)×¢ÏúSCSI TargetÄ£¿éÖеÄI£¯O¿ØÖÆÆ÷£¬Çå³ý¸Ã¿ØÖÆÆ÷µÄÉ豸IDºÅ£¬½«¸Ã¿ØÖÆÆ÷´ÓÈ«¾ÖÉÏÏÂÎÄÖÐÒÆ³ý¡£
(2)ÇåI£¯O¿ØÖÆÆ÷Ëù¿ØÖƵÄSCSIÃüÁÊÍ·ÅSCSIÃüÁîËùÕ¼ÓеÄÄڴ滺³åÇøµÈ×ÊÔ´¡£ÊÍ·ÅËùÕ¼ÓеÄÄڴ滺³åÇø¡£
(3)ÊÍ·ÅSRP TargetÄ£¿éÖÐI£¯O¿ØÖÆÆ÷ËùÕ¼ÓеÄ×ÊÔ´£¬Í£Ö¹¶ÔIB¶Ë¿Ú·þÎñÈ˿ڵÄ****²Ù×÷£¬ÊÍ·Å·þÎñÈë¿ÚËùÕ¼ÓеÄÄÚ´æ¿Õ¼ä×ÊÔ´£¬²¢¶ÔIB¶Ë¿Ú½øÐиüвÙ×÷£¬»Ö¸´¶Ë¿ÚµÄ³õʼÉèÖá£
(4)¶ÔËùÓÐRDMAͨµÀ½øÐжϿª²Ù×÷£®ÊÍ·ÅRDMAͨµÀËùÕ¼ÓеÄ×ÊÔ´¡£
(5)ÊÍ·ÅIB I£¯O¿ØÖÆÆ÷ËùÕ¼ÓеĿռ䡣





3 ʵÑé½á¹ûÓë·ÖÎö
3£®1 ʵÑé»·¾³
(1)Ó²¼þÅäÖãº
ÉñÍþSWIBS24ÐÍIBA½»»»»ú¡£
IBM System x3650·þÎñÆ÷3̨(Initiator01£¬Initiator02£¬Initiator03)¸÷ÅäÓÐÒ»¿éHCA¿¨£¬×÷ΪSRP Initiator¡£´æ´¢·þÎñÆ÷һ̨ÅäÖÃÒ»¿éHCA¿¨¡£×÷ΪSRP Target¡£
(2)Èí¼þÅäÖãº
ËĄ̈linux AS4 Updata4·þÎñÆ÷£®Äں˰汾Ϊ2£®6£®9-42¡£Initiator01¡¢Initiator02¡¢Initiator03°²×°IBAÇý¶¯³ÌÐòIBGD(InfiniBand Gold Distribution)µÄsRP Initiator£¬×÷ΪInitiator·þÎñÆ÷£»Target°²×°IBAÇý¶¯³ÌÐòOFED(OpenFabrics En-terprise Distribution)µÄSRP Target£¬×÷ΪTarget·þÎñÆ÷¡£±¾¿ÎÌâÒÀ¾ÝGPFSÖ§³ÖµÄÖ±½ÓÁ¬½Ó¡¢ÍøÂçÁ¬½ÓÁ½¸öģʽ£¬·Ö±ð¶Ô¶ÁдÐÔÄܽøÐбȽϡ£
Ö±½ÓÁ¬½Ó£ºÊ¹ÓÃSRPЭÒ飮ֻҪÇóÔÚInitiator01¡¢02¡¢03·þÎñÆ÷Éϰ²×°GPFS£¬TargetÉϲ»°²×°GPFS£¬Initiator·þÎñÆ÷ÓÉ1BÍøÂçͨ¹ýSRP Target»ñµÃRAID´æ´¢É豸ÐÅÏ¢£¬¼´¿É¶ÔÆä½øÐÐÖ±½Ó¹ÒÔØ¡¢¶Áд²Ù×÷¡£
ÍøÂçÁ¬½Ó£º²»Ê¹ÓÃSRPЭÒ飮ʹÓÃIPoIBЭÒé¡£ÒªÇóÔÚÿ¸ö·þÎñÆ÷É϶¼°²×°GPFS£¬Initiator·þÎñÆ÷²»ÄÜͨ¹ýSRPЭÒé»ñÈ¡Target·þÎñÆ÷ÉÏRAID´æ´¢É豸µÄÐÅ£¬g¡£InitiatorÖ»ÄÜͨ¹ýIPoIBÓëTargetÁªÏµ£¬Í¨¹ýTarget·þÎñÆ÷ÉϵÄGPFSÎļþϵͳ¶Ô´æ´¢É豸½øÐй²Ïí·ÃÎÊ¡£
3£®2 ʵÑé½á¹û
ϱíΪʵÑé½á¹û£¬IPoIBΪGPFSµÄÍøÂçÁ¬½Óģʽ£¬IB-SRPΪGPFSµÄÖ±½ÓÁ¬½Óģʽ£º
(1)Initiator01¡¢02¡¢03ͬʱ¶ÔTargetµÄRAIDÉ豸½øÐжÁд²Ù×÷(Èç±í1)¡£ÉèÖòâÊԵĿռä´óСΪ10 G£¬Ò»´Î¶Áд²Ù×÷µÄ¿é´óС·Ö±ðΪ64 k£¬128 k£¬256 k£¬ËÙ¶ÈÒÔK£¯sΪµ¥Î»¡£


10-01-15-91-1.jpg


(2)Initiator01¶ÔTargetµÄRAIDÉ豸µ¥¶À½øÐжÁд²âÊÔ(Èç±í2)¡£


10-01-15-36-1.jpg


3£®3 ½á¹û·ÖÎö
ʵÑé½á¹û±íÃ÷£¬ÔÚIBAÍøÂçÖÐʹÓÃSRPЭÒéµÄInitiatorºÍTargetÄ£¿é½øÐÐÊý¾Ý´«ÊäµÄËٶȱȵ¥´¿Ê¹ÓÃIBAÍøÂç½øÐÐÊý¾Ý´«ÊäµÄËٶȸ߳öºÜ¶à¡£
ʹÓÃSRPЭÒé½øÐÐÊý¾Ý´«Êäʱ£¬Initiator·þÎñÆ÷ÏòTar¡ªget·þÎñÆ÷µÄRAIDÉ豸´«ÊäÊý¾Ý£¬ÊÇÖ±½ÓÓëTargetÄÚ´æ½øÐÐÊý¾Ý´«Ê䣬ȻºóÓÉTargetµÄÄڴ潫Êý¾Ý´«µÝ¸øRAIDÉ豸¡£ÔÚ´Ëʱ£¬Targetϵͳ²¢²»Ö´ÐÐÊý¾ÝµÄ¿½±´¶¯×÷£¬²»¾­¹ýÄں˵ĵ÷Ó㬼õÉÙÁË´¦ÀíÍøÂçͨÐÅʱÔÚÄں˿ռäºÍÓû§¿Õ¼äÉÏÏÂÎÄÇл»µÄ´ÎÊý£¬Ïû³ýÁËÔÚÓ¦ÓÃÄÚ´æÓëÄÚºËÄÚ´æÖ®¼ä¸´ÖÆÊý¾ÝµÄÐèÒª£¬¼õÉÙÁËCPUµÄ¸ºÔØ¡£ÈƹýÄں˿ÉʹӦÓóÌÐò²»±ØÖ´ÐÐÄں˵÷ÓþͿÉÒÔÖ±½Ó·¢³öÃüÁî¡£¼õÉÙÁËÑÓ³Ùʱ¼ä£¬ÄÜÌÚ³ö×ÜÏ߿ռäºÍCPUÖÜÆÚÓÃÓڸĽøÓ¦ÓÃϵͳÐÔÄÜ¡£È»¶øÖ»ÀûÓÃIBAÍøÂç½øÐÐÊý¾Ý´«Êäʱ¡£TargetϵͳҪ½øÐÐÊý¾ÝµÄ¿½±´£®Äں˵ĵ÷Óã¬Ôö´óÁËCPUµÄ¸ºÔØ£®ÐÔÄÜÏà¶ÔϽµ¡£ËùÒÔ»ùÓÚSRPµÄ¶ÁдËÙ¶ÈÐÔÄÜ»á¸ü¸ß¡£



4 ½áÊøÓï
±¾ÎĽéÉÜÁËʵÏÖSRP TargetµÄ¹ý³Ì£®¶Ô¸÷¸ö¶ÀÁ¢¹¦Äܲ¿·ÖµÄ¹¤×÷Ô­Àí¡¢¹¤×÷»úÖÆÒÔ¼°¹¤×÷Á÷³Ì½øÐÐÁË·ÖÎö¡£²¢¸ø³öÁËIBA´æ´¢ÍøÂç·Ö±ðʹÓÃIPOIBЭÒéºÍSRPЭÒé¶ÁдÐÔÄܵIJâÊÔ½á¹û£¬¶ÔÓÚÉè¼ÆÊµÏÖ»ùÓÚIBAµÄ´æ´¢ÍøÂçÓÐ×ÅÖØÒªµÄ²Î¿¼¼ÛÖµ¡£

2010-4-20 12:53:17 »Ø¸´

ÒôƵ·ÖÀà´óÈ«

+ ²é¿´È«²¿·ÖÀà