IEBCOPY File Format¶
IEBCOPY is an IBM program with multiple uses, but for the sake of this
library we will talk about how it produces PDS unloads and their various
controls records as well as the directory structure.
IEBCOPY is used to unload partitioned datasets to a tape,
vitual tape, xmi, file etc. Its format is made up of controls records (COPYR1
and COPYR2), a listing of members and their metadata followed by the PDS
members file data.
COPYR1 Control Record¶
This record contains information about the file itself. It contains multiple items that can be used rebuild the PDS.
The COPYR1 record is 64 bytes long. Skipping the first 8 bytes this record
has the eye catcher of 0xCA6D0F which identifies it as a COPYR1 record.
After the eye catcher is the following information:
DS1DSORG - Dataset organization
DS1BLKL - Block size
DS1LRECL - Record length for all members
DS1RECFM - Record format
DS1REFD - Date last referenced
Multiple other fields are available in the COPYR1 record. Please refer to the following for more details: https://www.ibm.com/support/knowledgecenter/SSLTBW_2.2.0/com.ibm.zos.v2r2.idau100/u1322.htm
COPYR2 Control Record¶
Immediately following the COPYR1 control record is the COPYR2 record. This record contains information about the Data Extent Block (DEB) for the original dataset. This information is collected by this library but largely unused.
Directory Information¶
Following the control records is the directory information. This section is vairable length and can span multiple blocks. It contains the member names and metadata about the member. This section begins with:
00 00 00 00 00 00 00 00
or for PDSE:
08 00 00 00 00 00 00 00
and the key length, the length of the directory and the name of the last opened member.
After the header multiple entries exist, one for each member (a member is essentially a file) in this PDS. Each entry contains the following:
Member name
TTR, a pointer to the data for this member
Number of notes attached to this member
Alias flag, if enabled it means this member is an alias to another member
On top of this information optional information may be stored in the parameters field. This information is called “ISPF stats” since it is used mostly in ISPF when viewing and editing files. It can contain the following:
The file version, which can be automatically incremented by ISPF
Created date
Last modified date (down to the microsecond)
How many line the original file had
How many lines have been added
How many lines have been modified
The owner of the file
Member Data¶
After the directory block is the member data which can be broken down as:
Flag (1 byte)
Original “extent”, labeled as ‘M’ by IBM, (1 byte)
Binary number (2 bytes)
TTR
Data length
Followed by the member data.
More information about member data here: https://www.ibm.com/support/knowledgecenter/SSLTBW_2.2.0/com.ibm.zos.v2r2.idau100/u1327.htm
Metadata IEBCOPY¶
So what does this all look like on actual PDS? Using this library we can export
the metadata from the XMI file test_pds_msg.xmi in the tests folder:
{
"PYTHON.XMI.PDS": {
"COPYR1": {
"type": "PDS",
"DS1DSORG": 512,
"DS1BLKL": 27920,
"DS1LRECL": 80,
"DS1RECFM": "FB",
"DS1KEYL": 0,
"DS1OPTCD": 32,
"DS1SMSFG": 0,
"file_tape_blocksize": 3120,
"DVAOPTS": 12336,
"DVACLASS": 32,
"DVAUNIT": 15,
"DVAMAXRC": 32760,
"DVACYL": 10017,
"DVATRK": 15,
"DVATRKLN": 58786,
"DVAOVHD": 0,
"num_header_records": 2,
"DS1REFD": "210067",
"DS1SCEXT": "b'\\x80m\\x10'",
"DS1SCALO": "b'P\\x00\\x00\\x02'",
"DS1LSTAR": "b'\\x00\\x02\\x02'",
"DS1TRBAL": "b'\\x9f>'"
},
"COPYR2": {
"deb": "b'\\x01\\x00\\x00\\x00\\xff\\x00\\x00\\x00\\x8f\\x08\\x80\\x00\\x04\\x8b\\x00'",
"extents": [
"b'\\x01\\x00\\x00\\x00\\xff\\x00\\x00\\x00\\x8f\\x08\\x80\\x00\\x04\\x8b\\x00'",
"b'X\\xf4\\xe8X\\x00\\x00\\x01\\x0e\\x00\\x0b\\x01\\x0f\\x00\\x01\\x00\\x06'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'",
"b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'"
]
},
"members": {
"TESTING": {
"ttr": 8,
"alias": false,
"halfwords": 30,
"notes": 0,
"parms": "b'\\x01\\x00\\x00)\\x01!\\x06\\x7f\\x01!\\x06\\x7f\"S\\x00\\x02\\x00\\x02\\x00\\x00\\xd7\\xc8\\xc9\\xd3@@@@@@'",
"ispf": {
"version": "01.00",
"flags": 0,
"createdate": "2021-03-08T00:00:00.000000",
"modifydate": "2021-03-08T22:53:29.000000",
"lines": 2,
"newlines": 2,
"modlines": 0,
"user": "PHIL"
},
},
"Z15IMG": {
"ttr": 10,
"alias": false,
"halfwords": 0,
"notes": 0,
"parms": "b''",
"ispf": false,
}
}
Note
The actual raw member data has been omitted from this JSON output.
Notice that this file has a record format of Fixed Block (FB) and each line is
80 characters long. Also you can see that the first member TESTING contains
ISPF information whereas the second file Z15IMG does not.