How To Bypass A 'cookiewall' When Using Scrapy?

I'm a new user to Scrapy. After following the tutorials for extracting data from websites, I am trying to accomplish something similar on forums. What I want is to extract all pos

Solution 1:

In the end I found multiple ways to solve this problem:

• Simply having adding /?token=77c1f767bc31859fee1ffe041343fa48&allowcookies=ACCEPTEER+ALLE+COOKIES to the start url worked for this specific case
• I later switched to a CrawlSpider instead of a normal Spider, then I could add the xpath of the cookie button as the first rule.
• Clicking the button using the earlier mentioned Selenium also worked, but is a lot of hassle that is not really necessary...