- For the paragraph texts section, I am only able to get an empty list [ ] using the code r.html.find(“p”) after rendering. When using soup.find_all(“p”) with the lxml parser, it returns a result of errorTitle like “JavaScript is disabled” etc. Is that the reason why I am unable to get any text?
- Is it .difference only works for sets rather than lists? Is there an equivalent for lists?
Thanks a lot 🙂
1 Answer
Dear Vivian,
1. The BeautifulSoup approach indeed returns “JavaScript is disabled” because BeautifulSoup does not support JavaScript (more accurately, it is because of the initial request being made without a session to indicate the presence of JS tools).
What you can try is to pretend to be a browser, and then export the HTML file you received using requests-html to identify any problems.
2. The “.difference” method indeed only works for sets. There is no equivalent method for lists, however, this can easily be achieved with a list comprehension. E.g.
li_dif = [i for i in li1 + li2 if i not in li1 or i not in li2]
Best,
365 Team