How To Perform Http Get Operation In Python?

I want my code to take 'Request url' as input and gives output as 'Response XML'. This I want to achieve using python. I don't know how since I'm new to python. Although I know how

Solution 1:

from socket import *
s = socket()
s.connect(('example.com', 80))
s.send('GET / HTTP/1.1\r\n\r\n')
print s.recv(8192)

?

Or: http://docs.python.org/2/library/urllib2.html

import urllib2
f = urllib2.urlopen('http://www.python.org/')
print f.read(100)

---

The first option might need more header items, for instance:

from socket import *
s = socket()
s.connect(('example.com', 80))
s.send('GET / HTTP/1.1\r\nHost: example.com\r\nUser-Agent: MyScript\r\n\r\n')
print s.recv(8192)

Also, the first solution which i tend to lean towards (because, you do what you want and nothing else) requires you to have basic understanding of the HTTP protocol.

For instance, this is how the HTTP protocol works for a GET request:

GET <url> HTTP/1.1<cr+lf>
<header-key>: <value><cr+lf>
<cr+lf>

More on this here for instance: http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Client_request

Solution 2:

There is also a nice package called requests that makes your http needs in Python a lot easier.

For a get request you would do:

r = requests.get('http://www.example.com')
print(r.text)