Socialmetrica/Socialmetrica.package/NitterUser.class/instance/getPagesContentsFrom.Upto..st

19 lines
832 B
Smalltalk
Raw Normal View History

accessing
getPagesContentsFrom: anURL Upto: anInteger
"I retroactively get all pages contents until a specified page number.
TO DO: should this be splitted back to two methods, one getting the page urls and other its content?
or do we always be getting the cursor urls and its contents all the time.
[ ] Benchmark alternative approaches."
| response nextPageLink previousPageLink |
response := OrderedDictionary new.
response at: anURL put: (self documentTreeFor: anURL).
previousPageLink := anURL.
anInteger - 1 timesRepeat: [ | pageCursor |
pageCursor := self pageCursorFor:previousPageLink.
nextPageLink := self userNameLink, '/with_replies', pageCursor keys first.
response at: nextPageLink put: (XMLHTMLParser parse:nextPageLink asUrl retrieveContents).
previousPageLink := nextPageLink
].
^ response