Battle of Bots for Visa

With the advent of total digitalization, procedures such as online appointment scheduling have become quite common. However, many organizations still struggle to handle the influx of applications, leading to waiting times of months, if not years.

It’s understandable that queues and long waiting times often occur in places where resources are severely limited. However, even the system itself, which manages the electronic queue, is far from perfect.

Case of Visa Application

My wife and I decided to apply for tourist visas to a certain country. We prepared all the necessary documents and paid the fees. But here’s the problem: the next available appointment was scheduled for 1.5 years later. When you think about it, the country we wanted to visit as tourists is missing out on tourism revenue, and we’re not getting the experience we planned for.

There are many websites on the internet that offer to reschedule your appointment and significantly reduce the waiting time. How does it work? Someone simply created a bot that monitors available time slots, and with your personal information and access to your account, it reschedules your appointment to the nearest available time.

A fair question might arise: where do these available time slots come from? They could be due to the hiring of new employees (both temporary and permanent), cancellations or rescheduling by other applicants, or perhaps only a portion of the possible time intervals was initially made available for scheduling.

What can we assume about the work of visa centers?

Before moving forward, let’s take a moment to think on how the work of a visa center might be organized. I am not familiar with the actual organization, but from an outside perspective, I can make a few assumptions:

1. All public services provided by the visa center are standardized, which means the time allocated for each service is fairly consistent.
2. Visa centers offer services both through scheduled appointments and live queues. If one of the specialists responsible for appointments goes on vacation or falls ill, it is possible to either temporarily increase resources or redistribute them from the live queue (when making an appointment, the center commits to serving the client at a specific time, while no such commitment is made for the live queue).
3. If there is no live queue and no possibility to expand resources, it is initially easier (from the visa center’s perspective) to allocate a smaller number of available time slots to visit in order to mitigate risks and human factors.

Appointment scheduling issue

No one experiences negativity when they manage to schedule an appointment in the near future. We would be satisfied if we could see a dentist for an urgent appointment today and schedule a preventive check-up within a couple of weeks. The same goes for visas — it would be great to have a visa within a month, but certainly not after a year.

But here we are. We were able to schedule an appointment for a date that is 1.5 years from now.

Naturally, we are not willing to accept such a long waiting period, so I started looking for options to receive notifications about the availability of earlier time slots.

The birth of another bot

After googling and searching on GitHub, I managed to find several Python scripts that would notify you by email when a free date became available. The scripts were fairly simple, but I didn’t want to set up Python on my VPS. I decided to transfer the logic from the script to the tool I already had.

I’ve previously written about n8n (an open-source alternative to Zapier). The utility provides a wide range of possibilities, including the ability to schedule automations.

As it turned out, n8n, in addition to its extensive library of pre-built integrations, allows the installation of “community nodes.” Essentially, these are plugins supported by the community.

The visa center made the authentication process quite complicated by implementing reCAPTCHA and taking other measures that made it extremely difficult to write a bot using cURL. Therefore, I set up n8n + browserless integration, which allowed me to write bots using Pupeeter. Essentially, this tool launches Chrome in headless mode and allows interaction with the page as if it were opened in a browser. Consequently, all content is loaded as expected, and if a little randomness is added, even reCAPTCHA ceases to be a problem.

The bot grew quickly because n8n allows you to quickly build logic. The overall workflow looked something like this:

When a free time slot became available, my bot would send a notification to Telegram and then attempt to automatically book it. It also took into account various peculiarities of the specific visa center. For example, my account could be temporarily blocked, even if you check the time slots every ten minutes or so.

Battle of bots

Once the bot was up and running, it became apparent that there were several similar bots active on the center’s website. Free slots would appear regularly but disappear instantly. We could only rely on good timing and hope that our bot would be faster at some point.

Within a month, the bot found and booked a time slot, thus reducing the waiting time from 1.5 years to 4 weeks.

How it could have been different?

I started to think because such bots are a workaround. Due to the visa center’s failure to consider the clients’ needs, a small but understandable market for such bots emerged.

If we think again, we can get rid of this workaround at the booking logic level.

1. There are clients who are satisfied with the waiting time and a fixed appointment date and time. Nothing needs to be changed for them.
2. There are clients who would like to reschedule their booking to an earlier period, with specific preferences for the day of the week and time (for example, someone may work on alternate days without the possibility of missing a shift).
3. There are clients who would like to reschedule their booking to an earlier period and are willing to adapt.

In fact, the visa center could have organized an internal queue for those who wanted to reschedule and hidden the option to reschedule to earlier time slots from the website when they became available.

If a booking could be rescheduled, the center could send an email notification, and the visitor would have a X number of hours to click on a link to confirm the rescheduling (or decline it).

By implementing such a solution, the conditions for everyone wishing to visit the visa center would be the same, and there would be no point in developing bots.

Did the visa center save anything by not implementing such a logic? No. Due to the influx of bots, they had to introduce reCAPTCHA, implement custom token handling logic, enforce account blocking for frequent logins, and so on.

Is it difficult to implement rescheduling logic within the system? No. One could start with a simple choice:

• The booking date and time are satisfactory.
• I want to reschedule to an earlier period and am okay to change date and time.

Clients would then have a transparent interaction mechanism (to the extent possible).

Translated by ChatGPT