Description
I had experienced duplicate attendance activities when attendance records are fetched automatically from a ZKTeco biometric device using the Horilla scheduler.
The issue appears to occur specifically during scheduled/automatic biometric fetching. The same biometric punch can be processed multiple times, resulting in duplicate AttendanceActivity records with the exact same check-in timestamp.
had experience the same issue with the V1 at some point
Steps to Reproduce
na
Expected Behavior
A single biometric punch should result in one attendance event/activity, regardless of how many application workers are running.
The scheduler should also prevent the same biometric event from being processed more than once.
Actual Behavior
On 08/09/2026, several employees had multiple attendance activity records generated for the same biometric punch.
For example, one employee had a punch at:
05:13:39 UTC
Three AttendanceActivity records were created with exactly the same in_datetime:
Activity 1017
IN: 05:13:39 UTC
OUT: None
Activity 1024
IN: 05:13:39 UTC
OUT: None
Activity 1025
IN: 05:13:39 UTC
OUT: 14:22:34 UTC
The records were created within milliseconds of each other.
The application-level attendance record itself remained correct:
Check-in: 08:13
Check-out: 17:22
Worked: 09:09
However, the underlying AttendanceActivity table contained the duplicate activities.
I found 58 repeated punch groups for that day, with each repeated group containing three activities with the same in_datetime.
Another observation
The duplicate activities often consisted of:
two activities remaining open, and
one activity subsequently receiving the actual checkout.
This appears consistent with the same biometric punch being processed more than once.
The behavior also produced zero-duration activities in some cases because the existing open activity was closed using the same timestamp as the repeated check-in before another activity was created.
Screenshots
No response
Horilla Version / Branch
V2 / branch 2.0
Django Version
5.2
Python Version
3.1
Operating System
ubuntu linux 24
Browser
No response
Additional Information
While investigating the issue, I found that the ZKTeco scheduler was creating an APScheduler BackgroundScheduler() from the biometric application code.
The relevant pattern was essentially:
scheduler = BackgroundScheduler()
scheduler.add_job(
lambda: zk_biometric_attendance_scheduler(device.id),
"interval",
seconds=str_time_seconds(device.scheduler_duration),
)
scheduler.start()
With Gunicorn running multiple workers, this architecture potentially allows the scheduler to be instantiated independently by multiple application processes.
This may result in the same biometric device being polled multiple times concurrently, causing the same biometric event to be processed repeatedly.
I have not confirmed that this is definitively the root cause, so I would appreciate someone familiar with Horilla's scheduler architecture reviewing it.
Additional finding
The biometric fetch code uses the attendance timestamp / last-fetch time to determine which records to process, but I could not find any use of a unique biometric event ID such as the ZKTeco attendance record's uid to guarantee that an event is processed only once.
Therefore, if the same event is fetched by multiple scheduler executions, it appears possible for it to be inserted more than once.
What I changed as a temporary workaround
I moved the biometric polling into a single dedicated scheduler process instead of creating a separate BackgroundScheduler from the application/Gunicorn workers.
The application now has one central scheduler responsible for polling biometric devices.
After making this change, the scheduler is running successfully and the duplicate processing has not been observed in the subsequent records.
This is currently being treated as a workaround, not necessarily the final solution.
Possible Solution
Based on my investigation, the duplicate processing may be related to the biometric scheduler being initialized inside the Django/Gunicorn application process.
A possible solution would be to ensure that only one scheduler instance is responsible for polling biometric devices, rather than creating a BackgroundScheduler from each application worker.
For example, the biometric polling could be handled by a dedicated scheduler process:
Gunicorn workers
│
│ Web requests only
▼
Horilla
Dedicated Horilla scheduler
│
├── ZKTeco device 1
├── ZKTeco device 2
└── Other biometric devices
In addition, the biometric fetch operation could use the biometric device's unique event/record ID, where available, to make processing idempotent. This would provide a second layer of protection so that even if the same event is fetched more than once, it would only create one attendance activity.
In my installation, I tested the first approach by removing the per-device BackgroundScheduler initialization and moving biometric polling to a single dedicated scheduler process.
The scheduler is now executing once per interval without the previous multiple-processing behavior.
I would appreciate the Horilla team reviewing whether this approach should be incorporated into Horilla itself, and whether event-level deduplication should also be implemented for ZKTeco records.
Labels
bug
Priority
Medium
Assignees
No response
Related Issues
No response
Submission Checklist
Description
I had experienced duplicate attendance activities when attendance records are fetched automatically from a ZKTeco biometric device using the Horilla scheduler.
The issue appears to occur specifically during scheduled/automatic biometric fetching. The same biometric punch can be processed multiple times, resulting in duplicate AttendanceActivity records with the exact same check-in timestamp.
had experience the same issue with the V1 at some point
Steps to Reproduce
na
Expected Behavior
A single biometric punch should result in one attendance event/activity, regardless of how many application workers are running.
The scheduler should also prevent the same biometric event from being processed more than once.
Actual Behavior
On 08/09/2026, several employees had multiple attendance activity records generated for the same biometric punch.
For example, one employee had a punch at:
05:13:39 UTC
Three AttendanceActivity records were created with exactly the same in_datetime:
Activity 1017
IN: 05:13:39 UTC
OUT: None
Activity 1024
IN: 05:13:39 UTC
OUT: None
Activity 1025
IN: 05:13:39 UTC
OUT: 14:22:34 UTC
The records were created within milliseconds of each other.
The application-level attendance record itself remained correct:
Check-in: 08:13
Check-out: 17:22
Worked: 09:09
However, the underlying AttendanceActivity table contained the duplicate activities.
I found 58 repeated punch groups for that day, with each repeated group containing three activities with the same in_datetime.
Another observation
The duplicate activities often consisted of:
two activities remaining open, and
one activity subsequently receiving the actual checkout.
This appears consistent with the same biometric punch being processed more than once.
The behavior also produced zero-duration activities in some cases because the existing open activity was closed using the same timestamp as the repeated check-in before another activity was created.
Screenshots
No response
Horilla Version / Branch
V2 / branch 2.0
Django Version
5.2
Python Version
3.1
Operating System
ubuntu linux 24
Browser
No response
Additional Information
While investigating the issue, I found that the ZKTeco scheduler was creating an APScheduler BackgroundScheduler() from the biometric application code.
The relevant pattern was essentially:
scheduler = BackgroundScheduler()
scheduler.add_job(
lambda: zk_biometric_attendance_scheduler(device.id),
"interval",
seconds=str_time_seconds(device.scheduler_duration),
)
scheduler.start()
With Gunicorn running multiple workers, this architecture potentially allows the scheduler to be instantiated independently by multiple application processes.
This may result in the same biometric device being polled multiple times concurrently, causing the same biometric event to be processed repeatedly.
I have not confirmed that this is definitively the root cause, so I would appreciate someone familiar with Horilla's scheduler architecture reviewing it.
Additional finding
The biometric fetch code uses the attendance timestamp / last-fetch time to determine which records to process, but I could not find any use of a unique biometric event ID such as the ZKTeco attendance record's uid to guarantee that an event is processed only once.
Therefore, if the same event is fetched by multiple scheduler executions, it appears possible for it to be inserted more than once.
What I changed as a temporary workaround
I moved the biometric polling into a single dedicated scheduler process instead of creating a separate BackgroundScheduler from the application/Gunicorn workers.
The application now has one central scheduler responsible for polling biometric devices.
After making this change, the scheduler is running successfully and the duplicate processing has not been observed in the subsequent records.
This is currently being treated as a workaround, not necessarily the final solution.
Possible Solution
Based on my investigation, the duplicate processing may be related to the biometric scheduler being initialized inside the Django/Gunicorn application process.
A possible solution would be to ensure that only one scheduler instance is responsible for polling biometric devices, rather than creating a BackgroundScheduler from each application worker.
For example, the biometric polling could be handled by a dedicated scheduler process:
Gunicorn workers
│
│ Web requests only
▼
Horilla
Dedicated Horilla scheduler
│
├── ZKTeco device 1
├── ZKTeco device 2
└── Other biometric devices
In addition, the biometric fetch operation could use the biometric device's unique event/record ID, where available, to make processing idempotent. This would provide a second layer of protection so that even if the same event is fetched more than once, it would only create one attendance activity.
In my installation, I tested the first approach by removing the per-device BackgroundScheduler initialization and moving biometric polling to a single dedicated scheduler process.
The scheduler is now executing once per interval without the previous multiple-processing behavior.
I would appreciate the Horilla team reviewing whether this approach should be incorporated into Horilla itself, and whether event-level deduplication should also be implemented for ZKTeco records.
Labels
bug
Priority
Medium
Assignees
No response
Related Issues
No response
Submission Checklist