From nobody Sat May 14 19:21:14 2022 X-Original-To: freebsd-threads@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C9AE51AD5397 for ; Sat, 14 May 2022 19:21:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4L0wNf0Pyrz3tCl for ; Sat, 14 May 2022 19:21:21 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 24EJLEYj019584 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 14 May 2022 22:21:17 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 24EJLEYj019584 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 24EJLEZY019583; Sat, 14 May 2022 22:21:14 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 14 May 2022 22:21:14 +0300 From: Konstantin Belousov To: Paul Floyd Cc: freebsd-threads@freebsd.org Subject: Re: LDT/GDT and TCB use by threads Message-ID: References: <75f350fa-4fb5-f1e7-2a45-bb825a6620d3@gmail.com> List-Id: Threading List-Archive: https://lists.freebsd.org/archives/freebsd-threads List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-threads@freebsd.org X-BeenThere: freebsd-threads@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <75f350fa-4fb5-f1e7-2a45-bb825a6620d3@gmail.com> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4L0wNf0Pyrz3tCl X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-2.31 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.31)[-0.309]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-threads]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-ThisMailContainsUnwantedMimeParts: N On Sat, May 14, 2022 at 06:21:39PM +0200, Paul Floyd wrote: > Hi > > I'm trying to understand and fix a problem that I have with Valgrind (which > I maintain on FreeBSD). > > The full gory details are here > https://github.com/paulfloyd/freebsd_valgrind/issues/85 > > In short, on i386 Valgrind allocates a block of 8k slots for the LDT/GDT > when the first thread gets created. For each subsequent thread it uses up > one slot, until when the 8192 slots have all been used and libthr fails > because of a dirty thread exit. > > > On Linux x86, from what I've seen, main() allocates a thread area (Valgrind > intercepts a syscall sys_set_thread_area). Then syscall sys_clone (which is > used for both pthread_create and fork, with different flags) allocates a new > GDT every time. Thread exits deletes the thread area. And after a fork, some > Valgrind child code deletes  the GDTs other than for the running tid. > > My understanding is that FreeBSD has a lighter weight pthread > implementation. I think that I should be doing something similar in the case > of fork - delete all GDTs other than for the running tid. > > I'm not sure what I can do in the case of thread creation and exit. If I > delete the GDT like on Linux I get a crash since it seems on FreeBSD the GDT > still gets used after thr_exit() (in particular, I see calls to > get_currthread() in ld-elf32.so which access a lock in the TCB/GDT). > > I have a few questions (and may have more as I continue to debug). > > What exactly causes an application to switch to using threaded versions of > libc functions (I'm thinking specifically of fork())? Is it just by linking > with libthr? Or does the application need to create a thread first before > switching? For fork, just linking or dynamically loading libthr makes it use the libthr' intercepted version of the function. Start looking at lib/libc/sys/ interposing_table.c. For pthread stubs in libc, like pthread_mutex_lock(), there is similar, but different, stub overriding mechanism, see lib/libc/gen/_pthread_stubs.c for the initial pointer. Also, some places, like stdio, have direct check for __isthreaded value. > > Maybe a bit harder to say, but what is the usage of the GDT/ TCB in terms of > thread lifetime? And the same question for an ordinary process and for a > process created in a threaded application. GDT is static per-CPU. I do not remember i386 code, for amd64 we create fixed-size per-process LDT on the first LDT-related syscall, like sysarch(I386_SET_LDT). For i386 we might still dynamically grow LDT if needed. In long mode, %fs base points to TCB. In 32bit mode, %gs base does. On context switch, if %fs (or %gs) contain the magic segment values, amd64 kernel reloads the bases for incoming thread' values. If one of the %fs/%gs segment registers does not point to magic descriptor, it is left untouched (of course it is reloaded so that CPU has correct descriptor cache populated, but its base is not set).