How Do I Webscrape By Manipulating The Url? Python 3.5

I want to scrape a table of stock data from a website this table In my code, I generate an array of stock symbols. The URL for the website finviz generates tables for each particul

Solution 1:

You are passing a list to urllib.request.urlopen() instead of a string, that's all! So you were already really close.

To open all the different URLs, simply use a for loop.

for url in url_list:

    fpage = urllib.request.urlopen(url)
    fsoup = BeautifulSoup(fpage, 'html.parser')

    #scrape single page and add data to listwithopen('output.csv', 'wt') as file:
    writer = csv.writer(file)

    #write datalist

Solution 2:

You're passing a list to the urlopen method. Try the below and it'll retrieve data from the first URL.

fpage = urllib.request.urlopen(url_list[0])
fsoup = BeautifulSoup(fpage, 'html.parser')