[Bug 279011] deskutils/py-paperless-ngx: port affected by classifier training hanging

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=279011

            Bug ID: 279011
           Summary: deskutils/py-paperless-ngx: port affected by
                    classifier training hanging
           Product: Ports & Packages
           Version: Latest
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: Individual Port(s)
          Assignee: grembo@FreeBSD.org
          Reporter: freebsd.bugzilla@mail.tinsuke.com
          Assignee: grembo@FreeBSD.org
             Flags: maintainer-feedback?(grembo@FreeBSD.org)

The port seems to be affected by this issue:
https://github.com/paperless-ngx/paperless-ngx/discussions/2373

In short, on my instance, with 106 documents, 12 tag(s), 29 correspondent(s), 8
document type(s), 0 storage path(es), classifier training gets stuck/hangs
indefinitely with 1 CPU showing 100% usage.

The logs show:
[2024-05-15 15:56:20,696] [DEBUG] [paperless.classifier] Document
classification model does not exist (yet), not performing automatic matching.
[2024-05-15 15:56:20,697] [DEBUG] [paperless.classifier] Gathering data from
database...
[2024-05-15 15:56:20,843] [DEBUG] [paperless.classifier] 106 documents, 12
tag(s), 29 correspondent(s), 8 document type(s). 0 storage path(es)
[2024-05-15 15:56:20,877] [DEBUG] [paperless.classifier] Vectorizing data...
[2024-05-15 15:56:21,650] [DEBUG] [paperless.classifier] Training tags
classifier...
[2024-05-15 15:56:22,163] [DEBUG] [paperless.classifier] Training correspondent
classifier...

Running `su -l paperless -c 'paperless document_create_classifier'` shows:
OMP: Warning #96: Cannot form a team with 16 threads, using 1 instead.
OMP: Hint Consider unsetting KMP_DEVICE_THREAD_LIMIT (KMP_ALL_THREADS),
KMP_TEAMS_THREAD_LIMIT, and OMP_THREAD_LIMIT (if any are set).

As suggested in the linked discussion, adding this to /etc/profile fixes the
issue for me:
# Fix document classifier hanging:
#
https://github.com/paperless-ngx/paperless-ngx/discussions/2373#discussioncomment-9244780
export OMP_NUM_THREADS=1

The port could either set this variable as well (as the NixOS port did:
https://github.com/NixOS/nixpkgs/pull/299008), or try replacing the dependency
on openblas by mkl, but that seems to be problematic for FreeBSD:
https://github.com/paperless-ngx/paperless-ngx/discussions/2373#discussioncomment-8563927

-- 
You are receiving this mail because:
You are the assignee for the bug.