[FIXED] Using locust test in a GitLab CI pipeline, users never spawn

Issue

I try to run locust test using a GitLab CI, however, I faced an issue that users don't actually spawn, even though the corresponding message has been shown.

A simplified version of the .gitlab-ci.yml. Scenario is located in a separate file as @Cyberwiz kindly suggested in this question. Also, I tried to add scenario to the same locustfile.py with a test, however, it changed nothing.

 load_test:
   image: python:3.9-slim-buster
   before_script:
     - pip install virtualenv
     - virtualenv venv
     - source venv/bin/activate
     - pip install -r requirements.txt
     - locust -V
   script:
     - locust -H https://host -f locustfile.py,scenario.py --autostart --autoquit 0 --html locust_report.html

locustfile.py looks smth like this. I have a total of 10 users, each with unique email, and I login them one by one during 20 seconds. Added LOAD CLASS and USER AUTH prints to find out when these logins will happen in a pipeline (see the pipeline logs for that).

USER_EMAILS = [ ... emails ... ]
print(f"\nUSER EMAILS: {USER_EMAILS}")
 
 
class SocialLoad(TaskSet):
 
    print(f"\nLOAD CLASS") # for debugging
 
    def on_start(self):
        self.login()

    ...
    tasks
    ...

    def login():
        print(f"\nUSER AUTH")
        ...

GitLab pipeline logs. Looks like LOAD CLASS and USER AUTH prints occur after all the messages about users being spawned, which is weird to me. When I run this test without CI, they occur normally during the spawning process.

 [2022-10-04 20:14:24,657] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.main: Starting web interface at http://0.0.0.0:8089 (accepting connections from all network interfaces)
 [2022-10-04 20:14:24,666] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.main: Starting Locust 2.12.0
 [2022-10-04 20:14:24,666] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.runners: Shape test starting. User count and spawn rate are ignored for this type of load test
 [2022-10-04 20:14:24,671] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.runners: Shape worker starting
 [2022-10-04 20:14:24,671] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.runners: Shape test updating to 10 users at 0.50 spawn rate
 [2022-10-04 20:14:24,672] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.runners: Ramping to 10 users at a rate of 0.50 per second
 [2022-10-04 20:14:42,680] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.runners: All users spawned: {"User": 10} (10 total users)
 [2022-10-04 20:15:24,722] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.runners: Shape test stopping
 [2022-10-04 20:15:24,732] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.main: --run-time limit reached, stopping test
 [2022-10-04 20:15:24,732] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.main: --autoquit time reached, shutting down
 [2022-10-04 20:15:24,760] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.main: writing html report to file: locust_report.html
 [2022-10-04 20:15:24,761] runner-g-idswu7-project-4001-concurrent-0/INFO/locust.main: Shutting down (exit code 0)
 Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
 --------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
 --------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
          Aggregated                                                                         0     0(0.00%) |      0       0       0      0 |    0.00        0.00
 Response time percentiles (approximated)
 Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
 --------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
 --------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
 USER EMAILS: [ ... emails ... ]
 SOCIAL LOAD
 STARTED USER AUTH
 STARTED USER AUTH
 STARTED USER AUTH
 STARTED USER AUTH
 STARTED USER AUTH
 STARTED USER AUTH
 STARTED USER AUTH
 STARTED USER AUTH
 STARTED USER AUTH
 STARTED USER AUTH

Results are empty. Also, I printed out the creted locust_report.html file, which happened to be empty as well.

Question: even though the All users spawned: {"User": 10} (10 total users) message is shown, looks like users never spawn. Yes, there are print, which indicate that the login() function has been called, however, login itself (code after prints, which is absent here in snippets) never happens. If so, is it possible to make them to spawn 'normally' while test is running?

I think that the root cause is related to the retrieving emails step because when I run another simple test with only 1 user (and without retrieving any emails), all works smoothly.

Solution

It happened that the root cause for that was a VPN on the server side, it didn't include required IPs in its whitelist.

Answered By - Moveton