Python Logical Operation

I'm pretty new to python and I'm working on a web scraping project using the Scrapy library. I'm not using the built in domain restriction because I want to check if any of the lin

Solution 1:

or doesn't work that way. Try any:

if'domainName.com' in response.url and any(name in response.url for name in ('siteSection1', 'siteSection2', 'siteSection3')):

What's going on here is that or returns a logical or of its two arguments - x or y returns x if x evaluates to True, which for a string means it's not empty, or y if x does not evaluate to True. So ('siteSection1' or 'siteSection2' or 'siteSection3') evaluates to 'siteSection1' because 'siteSection1' is True when considered as a boolean.

Moreover, you're also using and to combine your criteria. and returns its first argument if that argument evaluates to False, or its second if the first argument evaluates to True. Therefore, if x and y in z does not test to see whether both x and y are in z. in has higher precedence than and - and I had to look that up - so that tests if x and (y in z). Again, domainName.com evaluates as True, so this will return just y in z.

any, conversely, is a built in function that takes an iterable of booleans and returns True or False - True if any of them are True, False otherwise. It stops its work as soon as it hits a True value, so it's efficient. I'm using a generator expression to tell it to keep checking your three different possible strings to see if any of them are in your response url.