🚸 Improve text extraction even further
This actually revamps the way the first paragraph is extracted from freesites and could cause a lot more descriptions to show up in Sone — which was the goal!
Previously I tried to locate all top-level nodes (under <body>) that themselves had text nodes below them and whose name did not start with an “h” (to exclude the header tags) but it turns out this can be easily defeated by wrapping all of the site in e.g. a <center> tag. And I’m sure that a <div> tag would do exactly the same…
So now I use a CSS selector query to get all <p> and <div> nodes, get those with text nodes below them and then get their text (which flattens them for me and removes embedded tags like <a> or <span>).
projects / Sone.git / commit