STFS: Difference between revisions

From Xenon Wiki
Jump to navigation Jump to search
imported>Sh042067
imported>Arkem
(Added py360 to tool list)
Line 735: Line 735:
XLAST inside the Xbox 360 SDK can create LIVE/PIRS packages, but it is illegal to share it.
XLAST inside the Xbox 360 SDK can create LIVE/PIRS packages, but it is illegal to share it.


A new python library [http://memeover.arkem.org/2011/04/py360-xbox-360-file-system-tools.html py360] can read STFS files
[[Category:File formats]]
[[Category:File formats]]
[[Category:Software]]
[[Category:Software]]

Revision as of 06:53, 17 April 2011

STFS (Secure Transacted File System) is the file system used by the Xbox 360 for all packages created and downloaded by the system. It is protected using a series of SHA1 hashes and a RSA signature. STFS is commonly found in Xbox 360 Content Packages (XContent), but is not limited to those only as the PEC (Profile Embedded Content) files employ STFS. The two known categories for STFS are read-only and writeable. Read-only content packages are found with a PIRS/LIVE signed header and writeable content packages are console signed (CON).

STFS

The 360 uses packages to transfer saves/content/games/pictures and more. Most packages start with the strings PIRS, LIVE or "CON ", some are even embedded inside gamer profiles (using the PEC format described below). All of these are STFS content packages which hold the real files along with metadata that the dashboard reads like the title, the licenses and the RSA signature which is used to verify the package.

The acronym STFS stands for Secure Transacted File System, which shows how the packages are secure (signature and hashes) and transacted (multiple file / directory revisions)

LIVE and PIRS files come from Xbox Live, these are signed using a private key that only Microsoft has. The console uses a public key which is hardcoded inside it to verify the package and make sure the person is allowed to use it. CON files are created by the console for saves and profiles. The console uses its own private key to sign CON files. Many editors are available for saves and profiles which can be used with no modification to the console.

Throughout an STFS package, there is a series of SHA1 hashes used to verify the package, and help with downloads (if a block isn't valid, it can be redownloaded). The hashes are located at certain parts of the file, a way of calculating where is (will be!) down below.

Volume Descriptor

Offset Length Type Information
0x0 0x1 byte Reserved
0x1 0x1 byte Block Seperation
0x2 0x2 signed short File Table Block Count
0x4 0x3 signed int24 File Table Block Number
0x7 0x14 bytes Top Hash Table Hash
0x1B 0x4 signed int Total Allocated Block Count
0x1F 0x4 signed int Total Unallocated Block Count

Content Packages

Header

Offset Length Type Information
0x0 0x4 ascii string Magic

The magic can be one of the following:

Signature Type Information
"CON " Signed by a console. Found on many files such as cache files, profiles, saved games.
PIRS Signed by Microsoft. Found on files delivered by Microsoft through non-Xbox Live means such as system updates.
LIVE Signed by Microsoft. Found on files delivered over Xbox Live such as items from the Marketplace like themes.

The following is used for Console Signed ("CON ") packages, and is used by the Xbox as the format for signatures generated by it:

Offset Length Type Information
0x4 0x2 bytes Public Key Certificate Size
0x6 0x5 bytes Certificate Owner Console ID
0xB 0x14 ascii string Certificate Owner Console Part Number
0x1F 0x1 byte Certificate Owner Console Type (1 for devkit, 2 for retail)
0x20 0x8 ascii string Certificate Date of Generation
0x28 0x4 bytes Public Exponent
0x2C 0x80 bytes Public Modulus
0xAC 0x100 bytes Certificate Signature
0x1AC 0x80 bytes Signature

and for remotely signed (LIVE/PIRS) packages:

Offset Length Type Information
0x4 0x100 bytes Package Signature
0x104 0x128 bytes Padding

The Package Signature is generated using the value at 0x32C (Content ID/Header SHA1).

Metadata

Offset Length Type Information
0x22C 0x100 license entries (see below) Licensing Data (used to check package owner)
0x32C 0x14 bytes Content ID / Header SHA1 Hash
0x340 0x4 unsigned int Entry ID
0x344 0x4 signed int Content Type (see below)
0x348 0x4 signed int Metadata Version (see below)
0x34C 0x8 signed long Content Size
0x354 0x4 unsigned int Media ID
0x358 0x4 signed int Version (system/title updates)
0x35C 0x4 signed int Base Version (system/title updates)
0x360 0x4 unsigned int Title ID
0x364 0x1 byte Platform (xbox/gfwl?)
0x365 0x1 byte Executable Type
0x366 0x1 byte Disc Number
0x367 0x1 byte Disc In Set
0x368 0x4 unsigned int Save Game ID
0x36C 0x5 bytes Console ID
0x371 0x8 bytes Profile ID
0x379 0x1 byte Volume Descriptor Struct Size (usually 0x24)
0x37A Volume Descriptor Struct Size STFS Volume Descriptor File System Volume Descriptor
0x39D 0x4 signed int Data File Count
0x3A1 0x8 signed long Data File Combined Size
0x3A9 0x8 bytes Reserved
0x3B1 0x4C bytes Padding
0x3FD 0x14 bytes Device ID
0x411 0x900 (each 0x80 = different locale) unicode string Display Name
0xD11 0x900 (each 0x80 = different locale) unicode string Display Description
0x1611 0x80 unicode string Publisher Name
0x1691 0x80 unicode string Title Name
0x1711 0x1 byte Transfer Flags (see below)
0x1712 0x4 signed int Thumbnail Image Size
0x1716 0x4 signed int Title Thumbnail Image Size
0x171A 0x4000 (thumbnail size) image Thumbnail Image
0x571A 0x4000 (title thumbnail size) image Title Thumbnail Image

Version 2

If the Metadata Version field is set to 2, the format is slightly changed:

Offset Length Type Information
0x3B1 0x10 bytes Series ID
0x3C1 0x10 bytes Season ID
0x3D1 0x2 signed short Season Number
0x3D5 0x2 signed short Episode Number
0x3D5 0x28 bytes Padding
0x171A 0x3D00 (thumbnail size) image Thumbnail Image
0x541A 0x300 (each 0x80 = different locale) image Additional Display Names
0x571A 0x3D00 (title thumbnail size) image Title Thumbnail Image
0x941A 0x300 (each 0x80 = different locale) image Additional Display Descriptions

License Entries

For every entry in the license data field:

Offset Length Type Information
0x0 0x8 signed long License ID (XUID / PUID / console id)
0x8 0x4 signed int License Bits
0xC 0x4 signed int License Flags

Content Types

Value Description
0xD0000 Arcade Title
0x9000 Avatar Item
0x40000 Cache File
0x2000000 Community Game
0x80000 Game Demo
0x20000 Gamer Picture
0xA0000 Game Title
0xC0000 Game Trailer
0x400000 Game Video
0x4000 Installed Game
0xB0000 Installer
0x2000 IPTV Pause Buffer
0xF0000 License Store
0x2 Marketplace Content
0x100000 Movie
0x300000 Music Video
0x500000 Podcast Video
0x10000 Profile
0x3 Publisher
0x1 Saved Game
0x50000 Storage Download
0x30000 Theme
0x200000 TV
0x90000 Video
0x600000 Viral Video
0x70000 Xbox Download
0x5000 Xbox Original Game
0x60000 Xbox Saved Game
0x1000 Xbox 360 Title
0x5000 Xbox Title
0xE0000 XNA

Transfer Flags

Value Description
0x00 DeviceID and ProfileID Transfer
0x20 Move Only Transfer
0x40 DeviceID Transfer
0x80 ProfileID Transfer
0xC0 None

Package Embedded Content (PEC)

When extra security is needed for content which is already using STFS, PEC files may be used to add an extra layer on top. PEC files use the STFS descriptor and algorithms, but has no similarity with content packages.

PEC files are only currently used for avatar items/clothing.

Block 0 starts at 0x3000, and hash table 0 at 0x1000/0x2000.

Header

Offset Length Type Information
0x0 0x228 Console Security Certificate Signed
0x228 0x14 bytes SHA1 hash from 0x23C-0x1000
0x23C 0x8 signed long Unknown
0x244 0x1 byte Volume Descriptor Struct Size (usually 0x24)
0x245 Volume Descriptor Struct Size STFS Volume Descriptor File System Volume Descriptor
0x268 0x4 signed int Unknown
0x26C 0x8 bytes Profile ID
0x274 0x1 byte Unknown
0x275 0x5 bytes Console ID

File Listing

The value at 0x37E (File Table Block Number on the structure above) determines where the file table begins. As it is a block number, you will have to convert it to an offset. Here is some code in C# for converting:

       public static int BlockToOffset(this int block)
       {
           int cluster = 0xc000 + block * 0x1000;
           while (block >= 170)
           {
               block /= 170;
               cluster += (block + 1) * 0x2000;
           }
           return cluster;
       }

Each embedded file starts at a 4096 byte boundary. The optional space between embedded files is filled with null bytes.

The file listing consists of entries which have the format below. The listing ends with an entry consisting of only null bytes.

Offset Length Type Information
0x0 0x28 ascii string File name, null-padded
0x28 0x1 byte Length of file name, plus flags
0x29 0x3 signed int24 Number of blocks allocated for file (little endian)
0x2C 0x3 signed int24 Copy of 0x29
0x2F 0x3 signed int24 Starting block number of file (little endian)
0x32 0x2 signed short Path indicator (big endian)
0x34 0x4 unsigned int Size of file in bytes (big endian)
0x38 0x4 signed int Update date/time stamp of file
0x3C 0x4 signed int Access date/time stamp of file

Byte 0x28 also has two flags: bit 6 and bit 7. The meaning of bit 6 is unknown, bit 7 indicates that the file is a directory.

The path indicator indicates the path of the file. -1 (0xFFFF) means that the file is in the root directory, any other value V refers to the (sub)directory which is listed as the Vth entry in the listing (counting from 0). Directories can nest.

The FAT format is used for the date/time stamps of the files.

Hash Tables / Block Offsets

A block is 0x1000 (4096) bytes and the first block is located at 0xC000. Every 0xAA (170) blocks there is a block that contains a hash of each block and every 0xAA*0xAA (0x70E4) blocks there is another table (presumably hashing the 0xAA table blocks). This means that when using a block number from a file listing you need to account for the hash tables which are not included in the block numbering system. For example block 171 is actually located where you would expect block 172 to reside ((171 + int(171/170)) * 0x1000 + 0xC000).

Offset Length Type Information
0x0 0x14 byte Hash(SHA1)
0x14 0x1 byte Status byte
0x15 0x3 byte Next block

Files are not always stored in consecutive blocks, you have to find the corresponding hash table record and read the next block number (similar to cluster maps in FAT).

Here is some C# code for converting a block to it's hash table position(it may not work perfectly!):

       public static int BlockToHashPosition(int block)
       {
           int multiplier = 0xAA;
           int res = block / 170;
           if (res != 0)
           {
               multiplier *= res;
           }
           else
               return 0xB000 + (block * 0x18);
           //For the BlockToOffset function, see above.
           int a = BlockToOffset(multiplier) + ((block % multiplier) * 0x18);
           return a - 0x1000;
       }

Tools

An (old) tool (Python 2.5 required) to analyze and extract these archive files is available at extract360.py (2008-08-03, 23056 bytes, MD5 = 3aa517c83d01c618927b78d0ca665d02)

wxPirs 1.1 can extract from LIVE/PIRS files fine, but as it doesn't use hash tables properly it doesn't work well with CON files.

A newer tool was released by DJ Shepherd called Le Fluffie, which can create and extract from CON/LIVE/PIRS files (but it has some problems with creation, some prefer to use XLAST)

XLAST inside the Xbox 360 SDK can create LIVE/PIRS packages, but it is illegal to share it.

A new python library py360 can read STFS files