question about fopen fd limit

[Ian Smith]

In freebsd-questions Digest, Vol 655, Issue 4, Message: 10
On Thu, 22 Dec 2016 11:53:08 +0800 ??? <hhsheng at corp.netease.com> wrote:
 > hi all,
 > 
 >    hi~
 >    we are from Chinese Game Develop Corp, Netease.
 >    and One of our product using FreeBsd as its OS platform.
 >    This Game has Millions of players online , and Each Server may 
 > holds 25000+ tcp connection at the same time.Thanks to BSD and kqueue 
 > :)
 > 
 >    for example, it's one of our server , netstat cmd to list 
 > connections overall...
 >    netstat -an | grep 13396 (it's our listening port) | wc -l
 >    23221
 > 
 >     recently we do some performance optimize and promote this connect 
 > limit to 28000+ or 30000+.
 >   But we find Freebsd has a limit that this huge online number will 
 > take 28000+ fd, and bsd FILE * struct's fd only support to SHORT . 
 > such as ..
 > 
 > struct __sFILE {
 >  ...
 >  short _file; /* (*) fileno, if Unix descriptor, else -1 */
 >  ...
 > 
 >   so if our server want to fopen some file when we still hold this 
 > online number, the fd amount may easily exceed 32767, and fopen 
 > definitely return a err code. then the server will appear some fataly 
 > ERROR.
 > 
 >   we do a simple test and confirm this situation.
 > 
 >   then in fopen's code , we notice that we can use open to return a 
 > fd instread of fopen to avoid this overflow, as below
 > 
 > 68 /* 
 > 1 * File descriptors are a full int, but _file is only a short. 
 > 2 * If we get a valid file descriptor that is greater than 
 > 3 * SHRT_MAX, then the fd will get sign-extended into an 
 > 4 * invalid file descriptor. Handle this case by failing the 
 > 5 * open. 
 > 6 */
 > 
 > 
 >   BUT ... so many c lib FILE series function needs a FILE * pointer 
 > as input argument, we can't convert all of them to fd, or it will be 
 > a rather suffering things to us.
 > 
 >   and even in BSD 10 , it seems this short limit still there , but 
 > other OS as debian , FILE strucnt's fileno is a int .
 >   
 >   we found an unoffical patch easily change this fileno to unsigned , 
 > but we are a very stready project, we can't afford the risk to use an 
 > unoffical patch.
 > 
 >   so, do you have any plan to change this fopen fd limit to UNSIGNED 
 > SHORT or int in the future ? ushort is enough for us .
 >   if you do , we are really glad and excited~~~~~~~if you don't ,it 
 > donen't matter, plz give us a reply so that we may need to
 >   find some other plan to resolved this suffering thing.
 >   LoL, thank you !!!!!
 > 
 > yours sincerely
 >                                        winson sheng

If you do not get a useful response here, I recommend reposting this to 
freebsd-net at freebsd.org where it will much more likely catch the eye of 
people who work with network code.

cheers, Ian