I have a really simple question: how much overhead is introduced by iSCSI and SCSI in the transfer of the data to read or write? I understand that this is done in the Data-In and Data-Out transport packets and I understand there is overhead from TCP/IP and Ethernet, but what additional overhead is needed by iSCSI/SCSI? I realize there is a full conversation that needs to happen between the initiator and target and I will ignore that part. So, I only want to know about the data packets sent following a read or write command. Thanks.
![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
Navigation
Install the app
How to install the app on iOS
Follow along with the video below to see how to install our site as a web app on your home screen.
Note: This feature may not be available in some browsers.
More options
-
Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
iSCSI protocol - basic question
- Thread starter buchs
- Start date
I have a really simple question: how much overhead is introduced by iSCSI and SCSI in the transfer of the data to read or write? I understand that this is done in the Data-In and Data-Out transport packets and I understand there is overhead from TCP/IP and Ethernet, but what additional overhead is needed by iSCSI/SCSI? I realize there is a full conversation that needs to happen between the initiator and target and I will ignore that part. So, I only want to know about the data packets sent following a read or write command. Thanks.
There isn't really a standard answer to that question. Overhead vs what? FCoE? IPoIB? It depends on your initiators and targets, processing power, offload etc. Are you talking theoretical (number of bytes in a packet header or encapsulated packet etc) or in practice? Also, are you looking at this from a storage management or a network management hat?
I am interested in the number of bytes added to pass some quantity of data - those bytes being overhead, i.e. they are not the data of interest to the end user/application, but are required as part of the SCSI/iSCSI protocol to do its job. I have the overhead for TCP/IP and those packets will be traveling via Ethernet. Since I am talking iSCSI, FCoE does not come into play and since I am talking TCP/IP/Ethernet, IPoIB does not come into play. I will assume that the target and initiator have sufficient processing power to handle what the iSCSI standard supports.
Here is what I have assembled from looking through the iSCSI RFC 3720 (www.ietf.org/rfc/rfc3720.txt).
The Protocol Data Unit (PDU - iSCSI "packet") has these elements:
1. Basic Header Segment - 48 bytes
2. Additional Header Segments - optional - 0 bytes (assume these are not needed)
3. Header Digest - 4 bytes (optional, but want this)
4. Data Segment - 8192 bytes (the max received data length is negotiated between the initiator and target, 8192 is the default, so will use that)
5. Data Digest - 4 bytes (optional, but want this)
So, I see an overhead of 56 bytes to transmit 8192 bytes of actual data.
@mwroobel, thanks for your reply. That document is a start, but leaves one still guessing on the lengths.
The Protocol Data Unit (PDU - iSCSI "packet") has these elements:
1. Basic Header Segment - 48 bytes
2. Additional Header Segments - optional - 0 bytes (assume these are not needed)
3. Header Digest - 4 bytes (optional, but want this)
4. Data Segment - 8192 bytes (the max received data length is negotiated between the initiator and target, 8192 is the default, so will use that)
5. Data Digest - 4 bytes (optional, but want this)
So, I see an overhead of 56 bytes to transmit 8192 bytes of actual data.
@mwroobel, thanks for your reply. That document is a start, but leaves one still guessing on the lengths.
Last edited:
Here is what I have assembled from looking through the iSCSI RFC 3720 (www.ietf.org/rfc/rfc3720.txt).
The Protocol Data Unit (PDU - iSCSI "packet") has these elements:
1. Basic Header Segment - 48 bytes
2. Additional Header Segments - optional - 0 bytes (assume these are not needed)
3. Header Digest - 4 bytes (optional, but want this)
4. Data Segment - 8192 bytes (the max received data length is negotiated between the initiator and target, 8192 is the default, so will use that)
5. Data Digest - 4 bytes (optional, but want this)
So, I see an overhead of 56 bytes to transmit 8192 bytes of actual data.
@mwroobel, thanks for your reply. That document is a start, but leaves one still guessing on the lengths.
You could have a data segment of 8192 (default) in an iscsi pdu. Depending on your initiator or target, MaxRecvDataSegmentLength generally sets the maximum DS size. Obviously bigger packets=fewer headers in general, but there are gotchas there too especially if you are in a heterogeneous network environment. Data digest is optional, the packets already have ethernet FCS crc-32 error checking and the DD is just an additional CRC. Whether you decide it is necessary is up to you. For more about performance tuning your packet MTU size you can look to http://www.ele.uri.edu/tcca/camera_ready/fujita.pdf
All the default and/or definable lengths are in the RFC, but in production environments they are/can be tuned based on usage realities and the hardware involved..
Last edited:
seems all my packets have 48bytes of iscsi overhead, in addition to tcp/ip/...
Read request packet of 106bytes (there can be more than one 48byte request per packet though)
Data packets
Read confirmation packet of 106bytes (same, more than one response per packet possible)
Writes are alittle different, they seem to only have a 32byte request header, then data for as many packets as needed
Then a write confirmation packet of 106bytes.
So basically, your looking at 64bytes per request overhead. No good way to calc tcp/ip overhead, unless you assume no requests are ever joined, but based on my packet inspections, lots of requests are joined.
Read request packet of 106bytes (there can be more than one 48byte request per packet though)
Data packets
Read confirmation packet of 106bytes (same, more than one response per packet possible)
Writes are alittle different, they seem to only have a 32byte request header, then data for as many packets as needed
Then a write confirmation packet of 106bytes.
So basically, your looking at 64bytes per request overhead. No good way to calc tcp/ip overhead, unless you assume no requests are ever joined, but based on my packet inspections, lots of requests are joined.