From nobody Sun Jan 08 07:44:47 2023 X-Original-To: dev-commits-doc-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NqTcc0FLBz2pTj0 for ; Sun, 8 Jan 2023 07:44:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NqTcb6RMsz4cC6; Sun, 8 Jan 2023 07:44:47 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1673163887; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fddkRiUvt6IhdaSKNLzF9dJrD89XdBh+f9V7sij7UG0=; b=Xn5oPJo5L6eKsvM8lyfBkV6MYUlyhtqiWqsjwk6ChBMHM1Q0lYvFrPaXmaBRqelTtpUlau tgjd3lgH8snAMNWo/JhUE72X6ZM/E/mnHn7n2pHWBtv3qzEvJIAFrXHy2DjHykgNKnmnEd JAVPWsLvR1XvaAIWY9AtSp/yPncn93TX3HjHNgQ5zr59XGvMmXVjcKGY4leHKas62/biQ7 Xx3U6tpgYzV7IEuk0dfFFhoU0ZiY/6LD6pQ8f99/v6ki4BGxX/Vd1MQT81yymem7/BUllb hTFB253BTPA0FEH7OOmC3Pa1EwRB7Qn27eDuvSu43z5xzxZ3YPghlyMVxNoTpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1673163887; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fddkRiUvt6IhdaSKNLzF9dJrD89XdBh+f9V7sij7UG0=; b=iBWbCaH8mqZe4GM23wEDIi1OLsvfHyKR+7vTNXImKAKlRE7AeMzLmjqd77dgMhqCjftYp3 gD8hROsWTn83iDpWGYQedoRUKctCEnfsa8r1NkpfTq0qtJOuzmVYd5QgXppqPeqwYsIV56 OK9cEK32o5k2y2UCfJ0MH3q3Giy4gf1QJ6gyxPjrwfQjGKdO6/a1XubhSoCLgc0t45BbXZ OQDsBOxkYenC50bUyLdVopOILtSUsiBsm6+9+gdQFd9ZnmwkUjorQKssyqiEVK0nRkT5Yp HHhZYRW15n5o+VaTJvInZGfTMgrqx20tbLZoxwLCk6OiJ7H0TZFPIqHhNT4B1Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1673163887; a=rsa-sha256; cv=none; b=ph7LvRNSupXnvwnq4UarD5KbOUeq6uvVUhH919J+1TzXgaLDS1V/i70g7czODvs60CG4Yz GXDF+oss3hauhxtT6yLhqchCBQHdzG5AabrKQuddm95F8Z9ZWFdOLgrcKaVopHyic8gCTP agEDz/ft7fe26GxRfNm607AhfbHGjJXBNG0M9aLevvL0PDwI8+EnLnY2h7T5rMokbBScxO DNbejioUo8PcjBPHI9GoAm4mg+vODVDLNw16cZTGbFs1nfTNQQ2R0ShCLtHfcBdy2S9IxJ CBnGhUhisqLBc+WkOsME25M+NY9pUy0V9hnqpOk/WTK6TeaUXWBoiSYv4fHxuQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4NqTcb5CPpzhPT; Sun, 8 Jan 2023 07:44:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 3087ilVp043230; Sun, 8 Jan 2023 07:44:47 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 3087ilGK043229; Sun, 8 Jan 2023 07:44:47 GMT (envelope-from git) Date: Sun, 8 Jan 2023 07:44:47 GMT Message-Id: <202301080744.3087ilGK043229@gitrepo.freebsd.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org From: Wolfram Schneider Subject: git: f02f9006ee - main - support opensearch autocomplete for FreeBSD manual pages List-Id: Commit messages for all branches of the doc repository List-Archive: https://lists.freebsd.org/archives/dev-commits-doc-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-doc-all@freebsd.org X-BeenThere: dev-commits-doc-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: wosch X-Git-Repository: doc X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: f02f9006ee45c59ae46b895ebf2b9446f58a88f2 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by wosch: URL: https://cgit.FreeBSD.org/doc/commit/?id=f02f9006ee45c59ae46b895ebf2b9446f58a88f2 commit f02f9006ee45c59ae46b895ebf2b9446f58a88f2 Author: Wolfram Schneider AuthorDate: 2023-01-08 07:38:48 +0000 Commit: Wolfram Schneider CommitDate: 2023-01-08 07:38:48 +0000 support opensearch autocomplete for FreeBSD manual pages --- website/content/en/cgi/man-autocomplete.cgi | 209 ++++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) diff --git a/website/content/en/cgi/man-autocomplete.cgi b/website/content/en/cgi/man-autocomplete.cgi new file mode 100755 index 0000000000..d05f1b0b23 --- /dev/null +++ b/website/content/en/cgi/man-autocomplete.cgi @@ -0,0 +1,209 @@ +#!/usr/local/bin/perl -T +# Copyright (c) 2009-2023 Wolfram Schneider, https://wolfram.schneider.org +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# man-autocomplete.cgi - autocomplete/suggestion service for FreeBSD manpages +# +# expected run time on a modern CPU: +# FreeBSD Release: 12ms for perl cgi script and 5ms for GNU grep => 17ms +# FreeBSD Release + Ports: 15ms for perl cgi script and 10ms for GNU grep => 25ms +# + +use lib qw(. ../../lib); +use MyCgiSimple; + +use strict; +use warnings; # 3% slower + +$ENV{PATH} = "/usr/local/bin:/bin:/usr/bin"; +$ENV{'LANG'} = 'C'; + +my $debug = 2; +binmode( \*STDIN, ":bytes" ); +binmode( \*STDOUT, ":bytes" ); +binmode( \*STDERR, ":bytes" ); + +sub suggestion { + my %args = @_; + + my $database = $args{'database'}; + my $icase = $args{'icase'}; + my $manpath = $args{'manpath'}; + my $query = $args{'query'}; + my $limit = $args{'limit'}; + + if ( !-e $database ) { + warn "$!: $database\n"; + return; + } + + # GNU grep, ripgrep, agrep, BSD grep etc. + my @command = ('grep'); + + # read more data for prefix match <=> sub-string match + my $limit_factor = 8; + + push @command, ( '-m', $limit * $limit_factor ); + push @command, '-i' if $icase == 1; + push @command, ( '--', $query, $database ); + + warn join( " ", @command ), "\n" if $debug >= 2; + if ( !open( IN, '-|' ) ) { + exec @command; + die "@command: $! :: $?\n"; + } + binmode( \*IN, ":bytes" ); + + my @data = (); + while () { + chomp; + s,["\s/],,g; + + # XXX: workaround for Firefox which ignores entries with "::" or "." inside a string + # Note: we have to undo this in man.cgi + s/::/: :/g; + s/\./ \./g; + push @data, $_; + + last if scalar(@data) >= $limit * $limit_factor; + } + + close IN; + + # a sorted list, but prefix matches first + # e.g. 'sort(1)' is before 'alphasort(3) if you searched for 'sor' + my $lc_query = $icase ? lc($query) : $query; + my @prefix = grep { index( lc($_), $lc_query ) == 0 } @data; + my @non_prefix = grep { index( lc($_), $lc_query ) != 0 } @data; + + # prefix first & real limit + @data = ( @prefix, @non_prefix ); + @data = splice( @data, 0, $limit ); + + warn "data: ", join( " ", @data ), "\n" if $debug >= 2; + return @data; +} + +sub escapeQuote { + my $string = shift; + + $string =~ s/"/\\"/g; + + return $string; +} + +# create devbridge autocomplete response JSON object +sub devbridge_autocomplete { + my $query = shift; + my $suggestion = shift; + + my @suggestion = @$suggestion; + + print qq/{ query:"/, escapeQuote($query), qq/", suggestions:[/; + print '"', join( '","', map { escapeQuote($_) } @suggestion ), '"' + if scalar(@suggestion) > 0; + print "] }\n"; + + warn "query '$query', suggestions: ", join ", ", @suggestion, "\n" + if $debug >= 1; +} + +# create opensearch autocomplete response JSON object +sub opensearch_autocomplete { + my $query = shift; + my $suggestion = shift; + + my @suggestion = @$suggestion; + + print '["', escapeQuote($query), '", ['; + + print qq{"}, join( '","', map { escapeQuote($_) } @suggestion ), qq{"} + if scalar(@suggestion) > 0; + print "]]\n"; + + warn "query '$query', suggestions: ", join ", ", @suggestion, "\n" + if $debug >= 1; +} + +###################################################################### +# param alias: query, q: search query +# manpath, m: release or release + ports +# icase,i : case sensitive +# debug, d: debug level + +my $max_suggestions = 24; +my $database_freebsd_release = + '/usr/local/www/bsddoc/man/etc/autocomplete/freebsd-release.txt'; +my $database_freebsd_release_ports = + '/usr/local/www/bsddoc/man/etc/autocomplete/freebsd-release-ports.txt'; + +my $q = new MyCgiSimple; + +my $test_street = "kurz"; + +my $query = $q->param('query') // $q->param('q') // $test_street; +my $manpath = $q->param('manpath') // $q->param('m') // ""; +my $d = $q->param('debug') // $q->param('d') // $debug; + +# we always use case insensive search for autocomplete +my $icase = $q->param('icase') // $q->param('i') // 1; + +$query = ( $query =~ /^(.+)$/ ? $1 : "" ); +$manpath = ( $manpath =~ /^([a-z\-]+)$/ ? $1 : "" ); +$icase = ( $icase =~ /^([01])$/ ? $1 : 0 ); +$debug = ( $d =~ /^([0-3])$/ ? $1 : $debug ); + +# not part of a filename +$query =~ s,["\s'/]+,,; + +my $database = + $manpath eq 'freebsd-release-ports' + ? $database_freebsd_release_ports + : $database_freebsd_release; + +my $expire = $debug >= 2 ? '+1s' : '+1h'; +print $q->header( + -type => 'text/javascript', + -charset => 'utf-8', + -expires => $expire, +); + +my @suggestion = (); +if ( length($query) >= 2 ) { + @suggestion = &suggestion( + 'database' => $database, + 'limit' => $max_suggestions, + 'query' => $query, + 'icase' => $icase, + 'manpath' => $manpath, + ); +} + +# ns=devbridge, for jQuery devbridge plugin +# &devbridge_autocomplete( $query, \@suggestion ); + +# ns=opensearch (Firefox etc.) +&opensearch_autocomplete( $query, \@suggestion ); + +#EOF