Re: Anybody seeing NextCloud crash?

On 14/11/2024 14:29, Ronald Klop wrote:
> Op 04-11-2024 om 05:39 schreef Kevin P. Neal:
>> I'm seeing a problem with NextCloud's php-fpm instance crashes when trying
>> to handle a request over the web interface. My web server is Apache 2.4
>> if it matters. The MacOS desktop sync client works just fine, and so does
>> the app from iOS 16 (I have an iPhone 8 that I'm upgrading soon).
>>
>> Apache responds with a "Service Unavailable" error page.
>>
>> In my /var/log/php-fpm.log I find this:
>>
>> [03-Nov-2024 11:06:45] WARNING: [pool www] child 68236 exited on signal 4 
>> (SIGILL) after 5.975134 seconds from start
> 
> 
> SIGILL means that an instruction was executed that was not understood by the CPU.
> This can be something like the software using SSE2 instructions and your CPU 
> does not support that.
> 
> I'm not sure if there are other reasons for SIGILL.

I recently learned about another reason and it's an instruction like ud2 on x86 
that can be inserted by a compiler for one reason or another (e.g., because of 
an assert).

But not in this case, I think.

Kevin correctly identified a (super)set of packages responsible for the problem.
The issue is known and it should be fixed in ports already.

See:
https://cgit.freebsd.org/ports/commit/?id=1876b07e4fcf269a1c57ac401ab57e4337bcf465

https://github.com/aous72/OpenJPH/releases/tag/0.18.0

> It helps if you could share what FreeBSD version your are running and on what 
> architecture.
> 
> The output of 'uname -a' gives this.
> 
> /var/run/dmesg.boot gives information about your CPU. This helps in determining 
> the supported instructions.
> Mine gives this:
> CPU: Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz (2400.14-MHz K8-class CPU)
>    Origin="GenuineIntel"  Id=0x406e3  Family=0x6  Model=0x4e  Stepping=3
>    
> Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
>    
> Features2=0xdeda2203<SSE3,PCLMULQDQ,SSSE3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,RDRAND,HV>
>    AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
>    AMD Features2=0x121<LAHF,ABM,Prefetch>
>    Structured Extended 
> Features=0x842421<FSGSBASE,AVX2,INVPCID,NFPUSG,RDSEED,CLFLUSHOPT>
>    Structured Extended Features3=0x10000400<MD_CLEAR,L1DFL>
>    TSC: P-state invariant
> 
> If php-fpm gives a core dump it is also possible to analyze on what instruction 
> it is failing. I don't know the command on how to do that by the top of my head. 
> Maybe somebody else can help.
> 
> If you can find out what package update resulted in the crash then you could ask 
> the package maintainer to compile it for older CPUs by default.
> 
>>
>> Every time I tried to reload a page on the web site I get a failure like
>> that one. The php-fpm processes are dying.
>>
>> I use the checkrestart package after every pkg upgrade and I restart
>> anything that gets mentioned. This handy utility isn't foolproof, but
>> I've had good luck with it.
>>
>> Reinstalling all my packages with pkg upgrade -f changed nothing.
>>
>> I rolled back my ZFS dataset to October 30's, rebooted (needed, don't know
>> why), and I have it back up and running now.
>>
>> I did "service php_fpm restart" after the bad update, but I forgot to check
>> that everything was still working. That's why I had to rollback so far --
>> I have SMS messages on my phone that imply it was working on that date.
>>
>> The problem was introduced with one of these updates:
>>
>> The following 11 package(s) will be affected (of 0 checked):
>>
>> New packages to be INSTALLED:
>>     openh264: 2.3.0,2
>>     openjph: 0.17.0
>>
>> Installed packages to be UPGRADED:
>>     ffmpeg: 6.1.2_4,1 -> 6.1.2_5,1
>>     glib: 2.80.5,2 -> 2.80.5_1,2
>>     libheif: 1.18.2_1 -> 1.19.1
>>     libnghttp2: 1.63.0 -> 1.64.0
>>     libssh2: 1.11.0_1,3 -> 1.11.1,3
>>     libvpx: 1.14.1 -> 1.15.0
>>     pciids: 20240920 -> 20241024
>>     py311-redis: 5.1.1 -> 5.2.0
>>     svt-av1: 2.2.0 -> 2.3.0
>>
>> Number of packages to be installed: 2
>> Number of packages to be upgraded: 9
>>
>> I do have my own poudriere setup, but I don't think I have any custom
>> settings at this point. I can doublecheck if anyone is curious, though.
>>
>> Anyone else seen this? Or should I file a bug report?
-- 
Andriy Gapon