#40_Improve_Preformance #41

Merged

silver merged 19 commits from #40_Improve_Preformance into main 2025-07-21 01:06:28 +00:00

Conversation 2 Commits 19 Files changed 24 +366 -257

silver commented 2025-07-20 23:55:57 +00:00

Owner

Copy link

While not exactly what was planned (18349f7bba/examples/e13_parallel_loops/src/main.rs (L48)) in #40 the end result is the same.
The scripts now wait for the cache to be populated before running.
Additionally some major changes to common/set_roles where some isseus with setting committee roles was not working as planned (trying to remove teh committee role from folks who did not have it)

Now the scripts run mostly under 10-15s each, allowing for greater ability to run them often withou incurring a time cost

Closes #40

While not exactly what was planned (https://github.com/serenity-rs/serenity/blob/18349f7bba43acad4261103eb38fe01d93f382df/examples/e13_parallel_loops/src/main.rs#L48) in #40 the end result is the same. The scripts now wait for the cache to be populated before running. Additionally some major changes to ``common/set_roles`` where some isseus with setting committee roles was not working as planned (trying to remove teh committee role from folks who did not have it) Now the scripts run mostly under 10-15s each, allowing for greater ability to run them often withou incurring a time cost Closes #40

silver added 18 commits 2025-07-20 23:55:57 +00:00

feat: cleaned up array that was used to count members/changes to a struct 2b2dfc2531

feat: setup update users to be every 5 min while using the cache 04aa0e63d4

feat: remove the update_users service 3dd81a5c54

fix: was calling the wrong ctx a8bed0bacc

fix: no need to pass in teh full object, a reference will do 43ef787d59

feat: drastically speed up the committee server script, it no longer tries to remove the committee role from those who dont have it. 1729ec0a54

fmt: better wording for logs 5815cde38b

feat: got the committee update running smoothly (using cache and far far faster due to fixing some logic issues) 96eb81293b

feat: removed the cronjob for updating the committee server eb88216740

fix: updating the wolves (user data) should not trigger a server update (directly). ... 13eb230754

That should always be triggered separately.
This was a holdover from a  time when updating teh users was expensive (timewise)

feat: moved the update data to the main thread 227db8a741

feat: moved the update server icon to the main thread feff293043

feat: restore teh original services, giving up the logging and control was too much 1af7f28a45

fix: these do not need to use teh cache bd9d0cd43f

fix: these were not using teh cache to access teh member/role data ... 6d08312f48

*was executing before teh cache was built (``cache_ready`` is only when teh cache has been init, not when it is populated)

fix: no longer needint to wait until the cache in teh main program is filled 57d4947edf

fmt: clippy and nightly fmt 9d409e3692

clippy: changes from nightly clippy ... d0726169ee

all of this is embeding teh var into teh format macro

silver added 1 commit 2025-07-21 00:00:02 +00:00

ci: improve the pipeline to test for the full suite 5feeb89bfd

silver added 1 commit 2025-07-21 00:11:10 +00:00

ci: dont pull in the lfs for teh PR pipeline build d82fcf4d37

silver added 1 commit 2025-07-21 00:17:32 +00:00

ci: checkout without LFS 65dd9a9d1a

silver commented 2025-07-21 00:21:39 +00:00

Author

Owner

Copy link

Preformance before

Committee taking 3 min

Jul 20 21:15:16 kitt systemd[1]: Starting Service for Skynet Discord Bot update_committee...
Jul 20 21:15:17 kitt update_committee[884942]: Skynet is connected!
Jul 20 21:15:17 kitt update_committee[884942]: Cache built successfully!
Jul 20 21:18:08 kitt systemd[1]: skynet_discord_bot-update_committee.service: Deactivated successfully.
...
Jul 20 22:15:16 kitt systemd[1]: Starting Service for Skynet Discord Bot update_committee...
Jul 20 22:15:17 kitt update_committee[886024]: Skynet is connected!
Jul 20 22:15:17 kitt update_committee[886024]: Cache built successfully!
Jul 20 22:18:08 kitt systemd[1]: skynet_discord_bot-update_committee.service: Deactivated successfully.
...
Jul 20 23:15:11 kitt systemd[1]: Starting Service for Skynet Discord Bot update_committee...
Jul 20 23:15:11 kitt update_committee[887094]: Skynet is connected!
Jul 20 23:15:11 kitt update_committee[887094]: Cache built successfully!
Jul 20 23:18:02 kitt systemd[1]: skynet_discord_bot-update_committee.service: Deactivated successfully.

Users taking a few sec anyways before these changes, seems it was jsut lucky with the cache being filled

Jul 21 00:05:27 kitt systemd[1]: Starting Service for Skynet Discord Bot update_users...
Jul 21 00:05:28 kitt update_users[888074]: Skynet is connected!
Jul 21 00:05:28 kitt update_users[888074]: Cache built successfully!
Jul 21 00:05:28 kitt update_users[888074]: 631559054862581800 Changes: New: +0, Current: +0/-0
Jul 21 00:05:29 kitt update_users[888074]: 689189992417067052 Changes: New: +0, Current: +0/-0
Jul 21 00:05:29 kitt update_users[888074]: 768921501138681869 Changes: New: +0, Current: +0/-0
Jul 21 00:05:30 kitt update_users[888074]: 1156591731698045040 Changes: New: +0, Current: +0/-0
Jul 21 00:05:30 kitt update_users[888074]: 1175193858766807100 Changes: New: +0, Current: +0/-0
Jul 21 00:05:30 kitt systemd[1]: skynet_discord_bot-update_users.service: Deactivated successfully.

Preformance before Committee taking 3 min ``` Jul 20 21:15:16 kitt systemd[1]: Starting Service for Skynet Discord Bot update_committee... Jul 20 21:15:17 kitt update_committee[884942]: Skynet is connected! Jul 20 21:15:17 kitt update_committee[884942]: Cache built successfully! Jul 20 21:18:08 kitt systemd[1]: skynet_discord_bot-update_committee.service: Deactivated successfully. ... Jul 20 22:15:16 kitt systemd[1]: Starting Service for Skynet Discord Bot update_committee... Jul 20 22:15:17 kitt update_committee[886024]: Skynet is connected! Jul 20 22:15:17 kitt update_committee[886024]: Cache built successfully! Jul 20 22:18:08 kitt systemd[1]: skynet_discord_bot-update_committee.service: Deactivated successfully. ... Jul 20 23:15:11 kitt systemd[1]: Starting Service for Skynet Discord Bot update_committee... Jul 20 23:15:11 kitt update_committee[887094]: Skynet is connected! Jul 20 23:15:11 kitt update_committee[887094]: Cache built successfully! Jul 20 23:18:02 kitt systemd[1]: skynet_discord_bot-update_committee.service: Deactivated successfully. ``` Users taking a few sec anyways before these changes, seems it was jsut lucky with the cache being filled ``` Jul 21 00:05:27 kitt systemd[1]: Starting Service for Skynet Discord Bot update_users... Jul 21 00:05:28 kitt update_users[888074]: Skynet is connected! Jul 21 00:05:28 kitt update_users[888074]: Cache built successfully! Jul 21 00:05:28 kitt update_users[888074]: 631559054862581800 Changes: New: +0, Current: +0/-0 Jul 21 00:05:29 kitt update_users[888074]: 689189992417067052 Changes: New: +0, Current: +0/-0 Jul 21 00:05:29 kitt update_users[888074]: 768921501138681869 Changes: New: +0, Current: +0/-0 Jul 21 00:05:30 kitt update_users[888074]: 1156591731698045040 Changes: New: +0, Current: +0/-0 Jul 21 00:05:30 kitt update_users[888074]: 1175193858766807100 Changes: New: +0, Current: +0/-0 Jul 21 00:05:30 kitt systemd[1]: skynet_discord_bot-update_users.service: Deactivated successfully. ```

silver added 1 commit 2025-07-21 00:26:53 +00:00

ci: see if using the ref directly will help afab2bed1c

silver added 1 commit 2025-07-21 00:32:12 +00:00

ci: test a slight modification 629db239e8

silver added 1 commit 2025-07-21 00:45:45 +00:00

ci: see if new get ldfs script works 233fc3008e

silver added 1 commit 2025-07-21 00:54:18 +00:00

ci: test using teh new v8 13ee1643c1

silver force-pushed #40_Improve_Preformance from 13ee1643c1 to 854e946a8f 2025-07-21 01:00:22 +00:00 Compare

silver merged commit 18fd45d39b into main 2025-07-21 01:06:27 +00:00

silver referenced this pull request from a commit 2025-07-21 01:06:32 +00:00

Merge pull request '#40_Improve_Preformance' (#41) from #40_Improve_Preformance into main

silver commented 2025-07-21 03:29:34 +00:00

Author

Owner

Copy link

Here is committee after, quite a drastic improvement

ul 21 03:20:23 kitt systemd[1]: Starting Service for Skynet Discord Bot update_committee...
Jul 21 03:20:24 kitt update_committee[902295]: Cache loaded
Jul 21 03:20:25 kitt update_committee[902295]: Cache built successfully!
Jul 21 03:22:05 kitt systemd[1]: skynet_discord_bot-update_committee.service: Deactivated successfully.
Jul 21 03:22:05 kitt systemd[1]: Finished Service for Skynet Discord Bot update_committee.
...
Jul 21 03:22:31 kitt systemd[1]: Starting Service for Skynet Discord Bot update_committee...
Jul 21 03:22:31 kitt update_committee[902338]: Cache loaded
Jul 21 03:22:32 kitt update_committee[902338]: Cache built successfully!
Jul 21 03:22:50 kitt systemd[1]: skynet_discord_bot-update_committee.service: Deactivated successfully.
Jul 21 03:22:50 kitt systemd[1]: Finished Service for Skynet Discord Bot update_committee.
...

Here is committee after, quite a drastic improvement ``` ul 21 03:20:23 kitt systemd[1]: Starting Service for Skynet Discord Bot update_committee... Jul 21 03:20:24 kitt update_committee[902295]: Cache loaded Jul 21 03:20:25 kitt update_committee[902295]: Cache built successfully! Jul 21 03:22:05 kitt systemd[1]: skynet_discord_bot-update_committee.service: Deactivated successfully. Jul 21 03:22:05 kitt systemd[1]: Finished Service for Skynet Discord Bot update_committee. ... Jul 21 03:22:31 kitt systemd[1]: Starting Service for Skynet Discord Bot update_committee... Jul 21 03:22:31 kitt update_committee[902338]: Cache loaded Jul 21 03:22:32 kitt update_committee[902338]: Cache built successfully! Jul 21 03:22:50 kitt systemd[1]: skynet_discord_bot-update_committee.service: Deactivated successfully. Jul 21 03:22:50 kitt systemd[1]: Finished Service for Skynet Discord Bot update_committee. ... ```

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

  

Trainee - Good First Issue

Something that is good for first issue

No labels

Trainee - Good First Issue

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Skynet/discord-bot#41

No description provided.