Loader forth changes for customization

Thu Nov 14 18:03:15 UTC 2013

On Nov 14, 2013, at 9:31 AM, Alfred Perlstein wrote:

> 
> On 11/13/13, 11:54 PM, Teske, Devin wrote:
>> On Nov 13, 2013, at 6:57 AM, Alfred Perlstein wrote:
>> 
>> 
>>> On 11/13/13, 12:18 AM, Teske, Devin wrote:
>>> 
>>>> On Nov 12, 2013, at 10:45 PM, Alfred Perlstein wrote:
>>>> 
>>>> 
>>>>> I added some hooks for menu.rc as well, you can see it via the github urls below.
>>>>> 
>>>>> I've attached a sample menu.rc.local that provides additional menus.
>>>>> 
>>>>> -Alfred
>>>>> 
>>>>> On 11/12/13, 6:35 PM, Alfred Perlstein wrote:
>>>>> 
>>>>>> Hey folks,
>>>>>> 
>>>>>> I added some forth using Devin's help to make it easier to customize the FreeBSD boot loader graphics.
>>>>>> 
>>>>>> Diffs are here:
>>>>>> 
>>>>>> https://github.com/alfredperlstein/freebsd/compare/loader_custom_rc
>>>>>> 
>>>>>> -or-
>>>>>> 
>>>>>> https://github.com/alfredperlstein/freebsd/compare/loader_custom_rc.diff
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Diff attached.
>>>>>> 
>>>>>> Also attached is a custom loader.rc file and loader.conf file that shows how to set the brand/logo.
>>>>>> 
>>>>>> Please review.
>>>>>> 
>>>>>> 
>>>> I signed up for a github account (thanks), and I started commenting on some lines.
>>>> 
>>> yay! :)
>>> 
>>>>>> -Alfred
>>>>>> 
>>>>> <menu.rc.local>
>>>>> 
>>>> Hmmm, I hadn't realized that you could say:
>>>> 
>>>> set foo=bar
>>>> 
>>>> Along-side setting functions in the same file.
>>>> 
>>>> I don't think you can set functions in an *.rc file, only in a *.4th file?
>>>> 
>>>> No? Maybe it's a false misconception of mine. I've been keeping them
>>>> separate for years. (but probably rightfully so, to keep *.rc files clean).
>>>> 
>>> It seems to work although I will talk to the team about making separate files for the set commands.
>>> 
>>> I've responded to your review comments here:
>>> 
>>> 
>>> https://github.com/alfredperlstein/freebsd/commit/0ca72dccd78b880b3e3ef4c2bb9ce025950a370b#commitcomment-4584862
>>> 
>>> 
>>> The changes I made are now in the branch
>>> 
>>> https://github.com/alfredperlstein/freebsd/tree/loader_custom_rc
>>>  and you likely should see them in the updated pull request I sent you.
>>> 
>>> 
>> I improved on a few things...
>> 
>> 
>> https://github.com/devinteske/freebsd/compare/freebsd:master...master
>> 
>> -or-
>> 
>> https://github.com/devinteske/freebsd/compare/freebsd:master...master.diff
>> 
>> -or-
>> Attached SVN patch.txt
>> 
> Hey this is really awesome.  I'll try to spin it up today and hopefully get it into FreeNAS/TrueOS today! 
> 
> I really like the level of comments here!  Having both the micro and macro explanation of what is going on is very helpful.
> 
> +: try-include ( -- ) \ see loader.4th(8)
> +  ['] include ( -- xt ) \ get the execution token of `include'
> +  catch ( xt -- exception# | 0 ) if \ failed
> +    LF parse ( c -- s-addr/u ) 2drop \ advance >in to EOL (drop data)
> +    \ ... prevents words unused by `include' from being interpreted
> +  then
> +; immediate \ interpret immediately for access to `source' (aka tib)
> +
> 
> 
> So a few questions here:
> If so when why are we clearing to EOL?

Technical aside (for later): ">in" represents the offset within the "sourcce" buffer.

When "include" aborts on error, the offset within the read buffer (>in) is not
advanced.

Technical aside #2: This is important because the interpreter picks up where
we left off (>in @ within source).

So, what's really going on is...

"include" aborts and there's no expectation that it will be caught and thus
no guard against the interpreter picking up where the [caught] abort left off.

In other words, "include" does not advance the text input buffer because it
aborts and thinks it has successfully "quit" the interpreter. But then we go and
catch it, so we need to advance the input stream ourselves.

In even more in-depth look... if you drop that code, here's what happens...

NB: In other words, simply : try-include ['] include catch drop ; immediate
NB: Don't use that implementation for the below reason.

try-include /boot/loader.rc.local
\ NB: All good if the file exists and "include" does not abort
try-include /does/not/exist
/does/not/exist: not found

In the above example, we see that the interpreter picked up at "/does/not/exist"
and tried to execute it as a Forth dictionary word.

Now... why clear all the way to the end of the line? Because include takes
multiple arguments...

	include file1 file2 file3 ...

Terminated by a newline. It parses using C so is more sophisticated than we
could do in a few lines of Forth...

	include /boot/loader".rc."local

NB: That works. And because we've chain-loaded to "include" from try-include,
that works with "try-include" as well.

>  Is "include" supposed to be alone on a line by itself?  You can't do this:
>   include file_that_exists.rc   5 6 + .
> and likewise you can't do:
>   try-include file_that_exists.rc   5 6 + .
>   try-include file_that_does_NOT_exists.rc   5 6 + .
> 

Correct... we've come across a limitation in the chain-loading of "include".
We have no way of knowing -- after "include" aborts -- how many words it
parsed.

But it's far worse than that.

We could make the assumption that it was parsing the last word... and in
the Forth support-functions (iirc) we have "parse-word", and even lower in
the built-in dictionary we have "word".

But here's the nasty edge-case. Remember that "include" uses C to parse
and is more sophisticated than we can do easily in Forth. So the following
syntax would break that assumption:

try-include "/foo   /bar"

The executed "include" would abort, leaving >in @ on the first double-quote
of "/foo   /bar", and if we assumed that it was parsing the last word, we would
call "parse-word" which would _only_ advance >in to the "/" in "/bar".

Hence why we read to the EOL -- for edge-cases like that where "include"
would have properly eaten the word (but does not advance >in prior to the
abort).

So I guess you could say that this is a work-around for the real solution which
would be to modify the FICL definition of "include" (which iirc should be in ficl.c)
to advance the offset to the end of the failed argument before calling abort...

But ... (geez, all these edge-cases) ... here's the final kicker.

Even if we modified "include" to advance to the end of the argument... it has
another bad habit. It aborts on the first bad file. So let's say you have:

try-include /good/file /bad/file /good/file

NOTE: /good/file exists and /bad/file does not.

When "include" gets to the /bad/file argument, it will abort, and let's say that
it did advance the input stream to (at the very least) the whitespace between
"/bad/file" and the last "/good/file".

At that point, the interpreter picks up at said whitespace and now you get:

	/good/file: not found

Because the interpreter will try to execute /good/file as a Forth word which is
naturally not in the dictionary.

If one wants to fix "include", what I surmise would need to be done is to
rewrite it to parse to EOL, advancing the input buffer prior to processing
any stored arguments. Then when we abort on the first error, if the abort
is caught the input stream is sufficiently advanced-in-offset that the
interpreter can pick up on the next line.

NB: So hence why try-include does this. It would be the same change for
include, but rather than changing the C code, I just made it in Forth (as a
single "LF parse 2drop" call to drain the input stream up to the EOL.

> It's not that important, just interesting.  I'm wondering though, with the exception of actually including the file or not, will both of these:
>   try-include file_that_exists.rc   5 6 + .

That would not print 11 because "include" would try to read "5" and get
an error. Then we (try-include) would return and drain the line.

>   try-include file_that_does_NOT_exists.rc   5 6 + .

This similarly would not print 11 because "include" would abort on the
file that doesn't exist, and "try-include" would drain the line.

> print '11'?  Or not?
> 

Neither would print 11, but ...

You wouldn't want to do this with "include" either.

For the very reason that "include" takes multiple arguments from the input
stream and would interpret your "5" as a file name.

If "5" is not a file, then it aborts and your code *seems* to work... but...
If "5" is a file and exists and does not cause an abort due to syntax...

NB: Assuming you catch the abort

The interpreter will pick up on "6" and you'll then execute "+" and get a
stack underflow.

You really do, whether using "include" or "try-include" want to break your
Forth code out onto a separate line.

Recap:
+ Forth on the same line as "include" will be tried as a filename first
+ Forth on the same line as "try-include" will be ignored (see previous
technical reasons above)

> Also, are there certain errors we want to report like "EISDIR" ?  I'm not concerned for my application, but it's an just academic question I have.
> 

Quite possibly. I think there's something in support.4th that will print an
error -- if not, we'll have to write it from scratch (and keeping to the maxim,
spin that out into something in support.4th where the IOR codes are
cleanly defined as constants).

> Finally thank you so much for the tutoring in forth, it's very cool and thanks for putting up with my obvious frustration at learning a new lang!
> 

No worries... it's fun isn't it? ;D
-- 
Devin

_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.