Commit Graph

175 Commits

Author SHA1 Message Date
7c6d7d9d80 Improving tweets scraping. 2022-05-15 12:39:10 -05:00
463781085b Fixing tweets filtering and write words for external cloud. 2022-05-15 12:31:23 -05:00
a515a24a81 Refactoring tweets scrapping: getting oldest ones. 2022-05-15 08:50:22 -05:00
a61de2ecb7 Refactoring tweets scrapping. 2022-05-15 08:23:21 -05:00
ac5d935ce1 Fixing messages for nitter user and tweets filtering. 2022-05-14 19:39:21 -05:00
390a6c3ef1 Bugfix: Avatar picture. 2022-05-14 18:17:34 -05:00
fa1cf4a6d8 Merge 7d6c8259f6 2022-05-14 17:46:54 -05:00
c16f109067 Bugfix: Tweets and avatar 2022-05-14 17:45:58 -05:00
defb1a8819 Modifyng installer for commons. 2022-05-14 16:56:07 -05:00
c32a351932 Dealing with changes in user Id. 2022-05-04 22:43:23 -05:00
459502cd5f Implementing reporting periods to limit messages processing inside a predefined timespan. 2022-05-04 07:27:25 -05:00
4abb9fe987 First draft to limit the reporting period. Future ones should have a TweetsCollection property to define such limits. 2022-05-03 23:08:54 -05:00
a2d5d80e57 Modifying newest tweet selection. 2022-05-03 21:25:22 -05:00
a08073232d Modifying newest tweet selection. 2022-05-03 20:50:20 -05:00
35d9d037d3 Modifying oldest tweet selection and oldest tweet for nitter user. 2022-05-03 12:32:05 -05:00
ae652fdf3a Modifying bars, quotes and replies histograms. 2022-05-03 11:05:56 -05:00
21b4f432c4 Temporal bugfix to deal with Twitter changes in user id. 2022-05-03 08:32:10 -05:00
419aa176ab Modifying date displays in histograms and fixing empty replies and quotes exporter. 2022-05-02 22:31:41 -05:00
6a628fddd3 Modyfing nitter user created at parsing. 2022-05-02 20:15:19 -05:00
aa5cf9cbba Modyfing nitter user sanitization for tex exporter. 2022-04-29 17:58:03 -05:00
104fe9e348 Fixing method for geting oldest tweets, modifying store avoiding duplication. 2022-04-28 16:17:56 -05:00
06b0d5748d Creating download tweets from oldest up to a page, modifying histograms exporters and external word cloud without mask. 2022-04-27 23:51:53 -05:00
ac963c753f Minor change in oldest tweet selection. 2022-04-27 15:51:40 -05:00
d2d0f739a8 Modifying nitter user for mustache template. 2022-04-27 15:38:10 -05:00
ad26072420 Fixing replies filtering and modifying tweets collection messages. 2022-04-26 14:05:58 -05:00
bd69b73529 Fixing raw tweets collection and tweets filtering from html. 2022-04-25 18:59:10 -05:00
b964460fa3 Fixing page contents downloader and modifying tweets collection. 2022-04-25 18:21:13 -05:00
a38124638d Creating tweet testing, and modifyng nitter user asdictonary. 2022-04-25 17:12:04 -05:00
3ff8dfc86a Code review: improved modularity. A second pass on #downloadWithRenaming: still pending. 2022-04-25 09:13:04 -05:00
bd279b1fd3 Code Review: Formatting and ward clause. Method should be made more modular. 2022-04-25 06:26:54 -05:00
bbfae14949 Creating exporter for report in LaTeX format. 2022-04-21 12:11:53 -05:00
f66d93e2ca Refactoring: minor changes in histogram exporters. 2022-04-20 16:59:57 -05:00
57adfca53a Creating quotes histogram exporter. 2022-04-20 16:44:34 -05:00
66298fc248 Fixing and creating retweets histogram exporters. 2022-04-20 16:27:54 -05:00
cde55509bf Creating retweets and quotes sorted by occurrences. 2022-04-20 15:49:50 -05:00
e840f733fe Tweet user bug fix. 2022-04-20 14:56:08 -05:00
1e06d16801 Removing unnecessary methods. 2022-04-20 14:13:02 -05:00
e43b375a6b Creating exporter for retweets. 2022-04-19 13:55:39 -05:00
4e6b134848 Merge 4ab8c6f0dd 2022-04-19 00:20:42 -05:00
99a03c0ad1 Tweets collection now inherits from OrderedCollection to improve reusability. 2022-04-19 00:19:23 -05:00
4ab8c6f0dd Recommiting changes in ee905 and two latter (why they were reverted?) 2022-04-18 22:36:10 -05:00
b92102a9d1 Merge dc945dd8f9 2022-04-18 22:16:00 -05:00
c2078d032f Minor fix in quotes scraping. 2022-04-18 22:14:17 -05:00
dc945dd8f9 Modifying histograms exporters. 2022-04-18 19:19:49 -05:00
2383ea4459 Refactoring collect tweets. 2022-04-18 19:00:58 -05:00
29c6deb617 Creating quotes filtering for nitter user messages. 2022-04-18 18:52:56 -05:00
3a58d281e2 Merge e98fd45083 2022-04-18 18:47:07 -05:00
a687e1b688 Creating metadata parsing from nitter html. 2022-04-18 18:44:52 -05:00
e98fd45083 Improving creation date processing/transfomation from different formats. #retrieveContents should also load local messages. 2022-04-18 18:36:52 -05:00
906b29a7ef Creating quotes metadata parsing. 2022-04-18 18:36:07 -05:00
8d87ee3ed7 Creating replies and retweets filtering and modifying histograms. 2022-04-18 17:29:09 -05:00
bfa2a2ab99 Modifying documentTree for downloading replies. 2022-04-18 16:14:01 -05:00
ad592beaf8 Creating replies download and recognition from nitter html. 2022-04-18 15:53:57 -05:00
49d5529934 Fixing creation date return depending on type of scrapping. 2022-04-17 12:30:28 -05:00
7de47a4959 Fixing tweets rendeting: loading users from local DB or remote scrapping. 2022-04-16 22:36:46 -05:00
ee9050a9b9 Fixing tweets rendeting by loading local users from DB. 2022-04-16 19:53:33 -05:00
50a3ef901b Code review: More modular code. 2022-04-16 12:08:30 -05:00
5d457e824a Creating histogram exporter for tweets and modifyng install external word cloud commons. 2022-04-16 01:59:52 -05:00
f28e884a79 Creating default exporter for tex template. 2022-04-16 01:37:11 -05:00
c9d9ec9620 Creating installation for template. 2022-04-16 01:32:02 -05:00
1f081f58cf Creating time spans division for tweets collection. 2022-04-15 18:53:06 -05:00
2de7cf1d11 Now external word cloud installs its common files and loads required data. 2022-04-15 14:35:11 -05:00
986f4a83bc Better variable namings 2022-04-15 08:49:20 -05:00
382dcd9954 Calculating time subperiods for tweets histogram. 2022-04-15 07:36:53 -05:00
0da5c06527 Initial methods for tweets histogram. 2022-04-15 00:05:17 -05:00
d61490e96c Making word cloud reproducible in local copy. 2022-04-14 20:27:26 -05:00
216832c3e2 Fixing external word cloud commons installation. 2022-04-14 20:00:06 -05:00
a50e37dce1 Modifying external cloud commons installation. 2022-04-14 19:39:46 -05:00
4341383a8c Modifying external word cloud parameters. 2022-04-14 19:20:45 -05:00
f97b193c5e Renaming methods for clarity. 2022-04-14 17:40:41 -05:00
b9eedc6ec3 Fixing connection to ReStore DB in tweets collection. 2022-04-14 16:43:31 -05:00
02ab2bd74a Local storage: getting messages from local DB. 2022-04-12 21:58:51 -05:00
f563f964b2 Creating testing for twitter user in ReStore. 2022-04-12 18:59:12 -05:00
8f2ed277fb Modifying tweet created for ReStore definition. 2022-04-12 18:23:08 -05:00
a445100e58 Fixing tweet form nitter html and collecting tweets for nitteruser. 2022-04-12 18:13:31 -05:00
bcf67809a8 Improving nitter user getting pages. 2022-04-12 16:05:59 -05:00
2c61055b38 Improved naming to better convey intention and more TO DO's. 2022-04-12 12:03:48 -05:00
52b90dd8dd New TO DO annotations and improving formating. 2022-04-12 10:15:24 -05:00
9b99ed0e33 Now each tweet stores metadata about the Nitter timeline where it appears. 2022-04-11 20:46:22 -05:00
db1a8e0502 Improving nitter user getting and parsing tweets. 2022-04-11 14:18:52 -05:00
bc168c4f0a Code review: Modularization. 2022-04-11 12:11:10 -05:00
65c36e924c Code review. Improving modularization and naming. 2022-04-11 11:42:54 -05:00
89a73603a3 Spliting between local and remote messages. Minor UI improvements. 2022-04-11 10:05:21 -05:00
2c2ee3b132 08.1376 Migration: Tweets previews problems deprecating BlLazyElement by BrAsyncWidget. 2022-04-10 12:37:25 -05:00
94ce588376 08.1376 Migration: Fixing profile previews problems caused by BlLazyElement deprecation. 2022-04-10 11:22:59 -05:00
01a62b2b2a 08.1376 Migration: Fixing profile previews problems caused by BlLazyElement deprecation. 2022-04-09 23:09:23 -05:00
36c261d534 Migrating to 08.1376, fixing code smells and refining UI elements because of external deprecation (but old problems arise). 2022-04-09 19:31:10 -05:00
a84e9504f9 Improving tweets parsing from nitter timeline item. 2022-04-08 23:50:31 -05:00
d9af46ec37 Improving tweet parsing from nitter timeline item. 2022-04-08 22:02:27 -05:00
e8f9e1ddd9 Merge pull request 'detach-recovery' (#1) from detach-recovery into master
Reviewed-on: https://code.tupale.co/Offray/Socialmetrica/pulls/1
2022-04-08 11:24:42 -05:00
907fffe4be Merge 2e6e480668 2022-04-08 11:17:21 -05:00
8c9e302fdd Fixing conflict. 2022-04-08 11:15:04 -05:00
ba05b8d0bb Changing profileImageUrl definition to ease ReStore serialization. Tip: Maybe derived keys can be used to serialize url as Strings in SQLite. 2022-04-08 11:10:24 -05:00
1e4f3d1190 Merge 2e6e480668 2022-04-07 14:20:57 -05:00
b198bde247 Improving urls for load more tweets and removing unnecessary method. ref 5b169c4ebf 2022-04-07 14:17:17 -05:00
2e6e480668 Merge 610c8e4ca1 2022-04-07 13:35:11 -05:00
17b39ec4ad Fixing TwitterUser id detection. 2022-04-07 13:33:46 -05:00
610c8e4ca1 Improving nitter user profile image exporter. 2022-04-07 11:28:18 -05:00
6a60f4f13f Code smell: redundant information of folder name and file name. 2022-04-06 13:24:51 -05:00
7bab421f80 Code review: Fixing some code smells. 2022-04-06 13:09:38 -05:00