[Bug 218922] Download of ports source not being identified as coming from FreeBSD OS.

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=218922

            Bug ID: 218922
           Summary: Download of ports source not being identified as
                    coming from FreeBSD OS.
           Product: Ports & Packages
           Version: Latest
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Many People
          Priority: ---
         Component: Ports Framework
          Assignee: portmgr at FreeBSD.org
          Reporter: qjail1 at a1poweruser.com
                CC: freebsd-ports-bugs at FreeBSD.org, portmgr at FreeBSD.org

It has come to my attention that the downloading of a ports source by our ports
framework is not identifying FreeBSD as the requesting operating system which
results in the download request being marked as "unknown" by the source-hosting
site.

Here is an example of what I am talking about.

https://sourceforge.net/projects/qjail/files/qjail-5.1.tar.bz2/stats/os?dates=2017-01-01%20to%202017-04-20

In discussion with sourceforge tech support, they said: 
“We unfortunately would not be able to determine the OS utilized if the
download request lacks the browser user agent information.

My port has its source hosted on sourceforge.net. That site has a function that
shows the count of times the port source file has been downloaded along with
the country and the type of operating system of the computer requesting the
download. 99.9% of the time the operating system is shown as "unknown". I can
only assume those are downloads originating from FreeBSD systems using the port
"make install" method. I believe whatever utility employed by the port system
framework to download the port source is not populating the HTTP_USER_AGENT or
the HTTP_REFERER environment variable, which is the root cause of this problem. 

I did this command on my host
/usr/bin/grep -r HTTP_REFERER /usr/ports/Tools
/usr/bin/grep -r HTTP_REFERER /usr/ports/Mk
/usr/bin/grep -r HTTP_ USER_AGENT /usr/ports/Tools
/usr/bin/grep -r HTTP_ USER_AGENT /usr/ports/Mk
Got no hits, but
/usr/bin/grep -r fetch /usr/ports/Tools
/usr/bin/grep -r fetch /usr/ports/Mk
Got a lot of hits.

I would say the ports system does not use HTTP_REFERER or HTTP_USER_AGENT and
the fetch utility is being employed to perform the physical download.

Here is my problem; I tried to read and follow the logic in some of those
scripts, but the coding style is foreign to me and I do not posses the
technical ability to manipulate the port system scripts to make changes. 

I think here is an untapped opportunity to let the world port source-hosting
servers take notice that FreeBSD is a widely used operating system. In its own
way it builds FreeBSD awareness and it’s not a big deal or time-consuming task
to accomplish.

Selecting the more correct download utility and populating the correct
environment variable with the value of “FreeBSD” would correct this problem. A
far better solution would be to inquire the host system for the version of the
host system OS and use that information, IE: FreeBSD 11.0-RELASE, which would
result in more meaningfully information being captured at the source-hosting
sites. 

I hope the ports powers-to-be can appreciate the benefits of this change.

-- 
You are receiving this mail because:
You are on the CC list for the bug.