Python Bytes Literal Has Extra Characters That Aren't Hex, But Alter The Value Of The String

I am used to the python byte literal syntax representing bytes as hex values such as b'\x7a' is the hex value 0x7a. I however have run into an issue that I don't know how it is wor

Solution 1:

Byte array notation converts certain values to their textual representation:

>>> b'\x12\x41\x42'b'\x12AB'

In this case the hex number 41 is the ASCII value for capital letter A.

Solution 2:

If the byte can be represented with a printable ASCII character that character is used instead of a hex escape, here is some code for generating a table of the representation for all byte values (columns are decimal, hex, and byte representation):

>>> all_bytes = bytes(range(256))
>>> for i inrange(256):
... print('{:<8}{:<#8x}{}'.format(i, i, all_bytes[i:i+1]))
... 00x0b'\x00'10x1b'\x01'20x2b'\x02'30x3b'\x03'40x4b'\x04'50x5b'\x05'60x6b'\x06'70x7b'\x07'80x8b'\x08'90x9b'\t'100xab'\n'110xbb'\x0b'120xcb'\x0c'130xdb'\r'140xeb'\x0e'150xfb'\x0f'160x10b'\x10'170x11b'\x11'180x12b'\x12'190x13b'\x13'200x14b'\x14'210x15b'\x15'220x16b'\x16'230x17b'\x17'240x18b'\x18'250x19b'\x19'260x1ab'\x1a'270x1bb'\x1b'280x1cb'\x1c'290x1db'\x1d'300x1eb'\x1e'310x1fb'\x1f'320x20b' '330x21b'!'340x22b'"'350x23b'#'360x24b'$'370x25b'%'380x26b'&'390x27b"'"400x28b'('410x29b')'420x2ab'*'430x2bb'+'440x2cb','450x2db'-'460x2eb'.'470x2fb'/'480x30b'0'490x31b'1'500x32b'2'510x33b'3'520x34b'4'530x35b'5'540x36b'6'550x37b'7'560x38b'8'570x39b'9'580x3ab':'590x3bb';'600x3cb'<'610x3db'='620x3eb'>'630x3fb'?'640x40b'@'650x41b'A'660x42b'B'670x43b'C'680x44b'D'690x45b'E'700x46b'F'710x47b'G'720x48b'H'730x49b'I'740x4ab'J'750x4bb'K'760x4cb'L'770x4db'M'780x4eb'N'790x4fb'O'800x50b'P'810x51b'Q'820x52b'R'830x53b'S'840x54b'T'850x55b'U'860x56b'V'870x57b'W'880x58b'X'890x59b'Y'900x5ab'Z'910x5bb'['920x5cb'\\'930x5db']'940x5eb'^'950x5fb'_'960x60b'`'970x61b'a'980x62b'b'990x63b'c'1000x64b'd'1010x65b'e'1020x66b'f'1030x67b'g'1040x68b'h'1050x69b'i'1060x6ab'j'1070x6bb'k'1080x6cb'l'1090x6db'm'1100x6eb'n'1110x6fb'o'1120x70b'p'1130x71b'q'1140x72b'r'1150x73b's'1160x74b't'1170x75b'u'1180x76b'v'1190x77b'w'1200x78b'x'1210x79b'y'1220x7ab'z'1230x7bb'{'1240x7cb'|'1250x7db'}'1260x7eb'~'1270x7fb'\x7f'1280x80b'\x80'1290x81b'\x81'1300x82b'\x82'1310x83b'\x83'1320x84b'\x84'1330x85b'\x85'1340x86b'\x86'1350x87b'\x87'1360x88b'\x88'1370x89b'\x89'1380x8ab'\x8a'1390x8bb'\x8b'1400x8cb'\x8c'1410x8db'\x8d'1420x8eb'\x8e'1430x8fb'\x8f'1440x90b'\x90'1450x91b'\x91'1460x92b'\x92'1470x93b'\x93'1480x94b'\x94'1490x95b'\x95'1500x96b'\x96'1510x97b'\x97'1520x98b'\x98'1530x99b'\x99'1540x9ab'\x9a'1550x9bb'\x9b'1560x9cb'\x9c'1570x9db'\x9d'1580x9eb'\x9e'1590x9fb'\x9f'1600xa0b'\xa0'1610xa1b'\xa1'1620xa2b'\xa2'1630xa3b'\xa3'1640xa4b'\xa4'1650xa5b'\xa5'1660xa6b'\xa6'1670xa7b'\xa7'1680xa8b'\xa8'1690xa9b'\xa9'1700xaab'\xaa'1710xabb'\xab'1720xacb'\xac'1730xadb'\xad'1740xaeb'\xae'1750xafb'\xaf'1760xb0b'\xb0'1770xb1b'\xb1'1780xb2b'\xb2'1790xb3b'\xb3'1800xb4b'\xb4'1810xb5b'\xb5'1820xb6b'\xb6'1830xb7b'\xb7'1840xb8b'\xb8'1850xb9b'\xb9'1860xbab'\xba'1870xbbb'\xbb'1880xbcb'\xbc'1890xbdb'\xbd'1900xbeb'\xbe'1910xbfb'\xbf'1920xc0b'\xc0'1930xc1b'\xc1'1940xc2b'\xc2'1950xc3b'\xc3'1960xc4b'\xc4'1970xc5b'\xc5'1980xc6b'\xc6'1990xc7b'\xc7'2000xc8b'\xc8'2010xc9b'\xc9'2020xcab'\xca'2030xcbb'\xcb'2040xccb'\xcc'2050xcdb'\xcd'2060xceb'\xce'2070xcfb'\xcf'2080xd0b'\xd0'2090xd1b'\xd1'2100xd2b'\xd2'2110xd3b'\xd3'2120xd4b'\xd4'2130xd5b'\xd5'2140xd6b'\xd6'2150xd7b'\xd7'2160xd8b'\xd8'2170xd9b'\xd9'2180xdab'\xda'2190xdbb'\xdb'2200xdcb'\xdc'2210xddb'\xdd'2220xdeb'\xde'2230xdfb'\xdf'2240xe0b'\xe0'2250xe1b'\xe1'2260xe2b'\xe2'2270xe3b'\xe3'2280xe4b'\xe4'2290xe5b'\xe5'2300xe6b'\xe6'2310xe7b'\xe7'2320xe8b'\xe8'2330xe9b'\xe9'2340xeab'\xea'2350xebb'\xeb'2360xecb'\xec'2370xedb'\xed'2380xeeb'\xee'2390xefb'\xef'2400xf0b'\xf0'2410xf1b'\xf1'2420xf2b'\xf2'2430xf3b'\xf3'2440xf4b'\xf4'2450xf5b'\xf5'2460xf6b'\xf6'2470xf7b'\xf7'2480xf8b'\xf8'2490xf9b'\xf9'2500xfab'\xfa'2510xfbb'\xfb'2520xfcb'\xfc'2530xfdb'\xfd'2540xfeb'\xfe'2550xffb'\xff'

Solution 3:

When calling repr() on a bytestring it tries to show bytes which represent usual ASCII characters as such. Only the "unusual" ones are shown as \x..