Socialmetrica/Socialmetrica.package/NitterUser.class/instance/getPagesContentsUpto..st

19 lines
870 B
Smalltalk

accessing
getPagesContentsUpto: anInteger
"I retroactively get all pages contents until a specified page number.
TO DO: should this be splitted back to two methods, one getting the page urls and other its content?
or do we always be getting the cursor urls and its contents all the time.
[ ] Benchmark alternative approaches."
| response nextPageLink previousPageLink |
response := OrderedDictionary new.
response at: (self userNameLink, '/with_replies') put: self documentTree.
previousPageLink := (self userNameLink, '/with_replies').
anInteger - 1 timesRepeat: [ | pageCursor |
pageCursor := self pageCursorFor:previousPageLink.
nextPageLink := self userNameLink, '/with_replies', pageCursor keys first.
response at: nextPageLink put: (XMLHTMLParser parse:nextPageLink asUrl retrieveContents).
previousPageLink := nextPageLink
].
^ response