News - 2022/07/31 - FLL 0.6.0 Release
Author: Kevin Day
Description: News post on 2022/07/31.
Keywords: Kevin Day, Kevux, FLL, Featureless, Linux, Library, Distribution, Open-Source, News, 2022

# 2022 / 07 / 31 - FLL 0.6.0 Release

The Featureless Linux Library stable version `0.6.0` is released.

Several new programs are introduced since the `0.4.0` stable release series:

- control
- controller
- fss_embedded_list_read
- fss_embedded_list_write
- fss_extended_list_write
- fss_identify
- fss_payload_read
- fss_payload_write
- iki_write
- utf8

Most notable of these programs are the controller program and the utf8 program.

The controller program is a re-interpretation of the Featureless Make program written in a more generalized fashion that is focused on performing a set of tasks. This is very similar to a system initialize program such as sysvinit. Support for running as a system initialize replacement is partially complete. There is currently no way to send messages to the controller program to start and stop anything. The control program, which is used to communicate to the controller program is also incomplete. The next FLL development verrsion, the `0.7.0` series, will heavily focused on network protocols.

The utf8 program is a handy tool to convert from UTF-8 characters to their Unicode value or vice-versa. This can convert entire files.

Featureless Make has some major changes. The fake program now accepts piped data, which must be a valid fakefile. There are new Objects available for the build settings and the fakefile.

The default behavior of Featureless Make is now more similar to the GNU make program. When running `fake` with no parameters, it will first search for a `fakefile` in the current directory before checking for one at `data/build/fakefile`. This is analogous to how running `make` with no parameters will search for a `Makefile` in the current directory.

One major change is the complete rewrite of the printing functionality. The new design is heavily focused around formatted printing, similar to the `printf()` family of functions. Unlike the standard `printf()` family of functions, the new formatted printing understands the FLL special structures such as dynamic strings. The new formatted printing is UTF-8 aware and can handle printing raw strings, printing safe strings, and printing NULL characters. A safe string is a string where control codes or invalid Unicode sequences are replaced with place holders.

That standard console options have changed. All output changing parameters have had their short parameter argument made upper case. For example, `+q` is now `+Q`.

A new `+E/++error` parameter is introduced. This new parameter acts almost identical to `+Q/++Quiet`. The difference is that error messages are still printed.

The documentation may be found with the source code. Much of this website has been updated to also provide most of this documentation as well. The API is currently only available in the source code via the header files.

A major improvement of this release is safer and more stable code. This is thanks to a massive amount of time and effort spent into writing unit tests. The unit tests process is performed using the Featureless Make program. In addition to the unit tests, the static analyzer from GCC has been used to better identify and resolve problems.

The following are changes since the 0.5.10 development release.

Security Fixes:

- FLL: Environment value has invalid read in f_environment_set().
- FLL: Invalid read for formatted printing using partial ranges on a string.
- FLL: Invalid reads on NULL terminated strings with a max width greater than 1.
- Featureless Make: Invalid read when using -s/--settings.
- Featureless Make: Invalid read when processing not condition minimum value.
- Featureless Make: Invalid read when processing an empty fakefile.

Features:

- FLL: Add missing function f_environment_get_all().
- FLL: Add missing signal functions f_signal_action(), f_signal_pause(), and f_signal_suspend().
- FLL: Add support for +E/++error and change '+q' to '+Q'.
- Controller: Expose the "define" and "parameter" at the Entry and Exit level.
- Featureless Make: Now supports a piped fakefile.
- Featureless Make: Now supports the "write" operation.
- Featureless Make: Now supports to the "print" operation for escape sequences just like the "write" operation has.
- Featureless Make: Now supports the "top" parameter.
- Featureless Make: Now supports the "current" parameter.
- Featureless Make: Improve user experience by defaulting specified fakefile and build settings file to present working directory.

Bug Fixes:

- FLL: IKI variables are incorrectly being processed when there is a non-IKI IKI-like string.
- FLL: Single quotes are not being properly detected in FSS Extended Read functions.
- FLL: private_f_file_open_at() has an incomplete set if ifdef conditions.
- Byte Dump: Parameter after either the -N or -W is being processed when it should not be.
- Controller: Incorrect information is printed on certain errors.
- Controller: Delimits are not being applied for IKI variables.
- Controller: "State is now ..." should not be printed when quiet is passed.
- Controller: Guarantee that the child processes are terminated.
- Controller: IKI substitution is incorrectly applied.
- Featureless Make: Condition blocks are still being processed when they should be skipped.
- Featureless Make: Empty strings improperly pass validation checks.
- Featureless Make: IKI variable substitution results in incorrect parameter structure.
- Featureless Make: Error verbosity is not being preserved.
- Featureless Make: Text is being incorrectly stripped out.
- Featureless Make: The copy operation doesn't work properly with trailing slashes.
- Featureless Make: Copy should destination should be the base name rather than the entire path.
- Featureless Make: Link operation is incorrect, support new link arguments, and clone, copy, and move operations failure problems.
- Featureless Make: Block is incorrectly being reset when an else condition precedes an if condition.

Refactors:

- Controller: Use "settings" instead of "setting" for better consistency between fake program and controller program.
- Featureless Make: Rename "script" to "engine" to avoid possible confusion with the different uses of "script".
- Featureless Make: "if defined parameter" and similar are simplified into something more like "if define" and "if parameter".
- Featureless Make: Rename "exists" to "exist".

Updates:

- FLL: Change execute error code handling to better accommodate standard GNU Bash return codes.
- FLL: Further reduce memory allocation pressure by increasing default small allocation.
- FLL: Restrict environment to PATH and LD_LIBRARY_PATH by default in all FLL build settings files.
- FLL: The standard program setup should explicitly unblock all signals other than the default
- FLL: Clarify IKI standard in regards to delimits and improve specification file.
- FLL: Improve specification files.
- FLL: The f_color needs to follow latest practices with private functions.
- Controller: Change default controller settings path to './'.
- Controller: Strip out NULL characters after applying delimits.
- Controller: Timeout changes.
- Featureless Make: Make environment variable handling design consistent between fake and controller programs.
- Featureless Make: Ensure first if block is initialized to operate.
- Featureless Make: Replace Unicode Terminate escape sequence with Non-printing escape sequence.
- Featureless Make: Reduce number of allocations by pre-determining size when easily possible.
- Featureless Make: Organize the validate operation into separate type functions and improve error handling logic.
- Featureless Make: The example bzip2 now supports running the tests.

Check out the project on the FLL release page.

The project is built like a tool chest. Individuals who are unfamiliar with the project should have an easier time trying out the pre-packaged monolithic sources. These programs, by default, are designed to build against a library built using the monolithic source tree.

Kevin Day

## Example Bzip2 Makefile

Bzip2 has a simple `Makefile` that is used here as an example of the differences between a `Makefile` and a `fakefile`.

This is taken straight out of the Bzip2, version 1.0.8, source code and is under its own separate copyright. This is file has a copyright of: Copyright (C) 1996-2019 Julian Seward. The use here as an example should be technically under the Fair Use laws. Just in case, the entire file is provided with the copyright comments with it. The Bzip2 source code may be found at https://www.sourceware.org/bzip2/.

    # ------------------------------------------------------------------
    # This file is part of bzip2/libbzip2, a program and library for
    # lossless, block-sorting data compression.
    #
    # bzip2/libbzip2 version 1.0.8 of 13 July 2019
    # Copyright (C) 1996-2019 Julian Seward
    #
    # Please read the WARNING, DISCLAIMER and PATENTS sections in the
    # README file.
    #
    # This program is released under the terms of the license contained
    # in the file LICENSE.
    # ------------------------------------------------------------------

    SHELL=/bin/sh

    # To assist in cross-compiling
    CC=gcc
    AR=ar
    RANLIB=ranlib
    LDFLAGS=

    BIGFILES=-D_FILE_OFFSET_BITS=64
    CFLAGS=-Wall -Winline -O2 -g $(BIGFILES)

    # Where you want it installed when you do 'make install'
    PREFIX=/usr/local

    OBJS= blocksort.o  \
          huffman.o    \
          crctable.o   \
          randtable.o  \
          compress.o   \
          decompress.o \
          bzlib.o

    all: libbz2.a bzip2 bzip2recover test

    bzip2: libbz2.a bzip2.o
      $(CC) $(CFLAGS) $(LDFLAGS) -o bzip2 bzip2.o -L. -lbz2

    bzip2recover: bzip2recover.o
      $(CC) $(CFLAGS) $(LDFLAGS) -o bzip2recover bzip2recover.o

    libbz2.a: $(OBJS)
      rm -f libbz2.a
      $(AR) cq libbz2.a $(OBJS)
      @if ( test -f $(RANLIB) -o -f /usr/bin/ranlib -o \
        -f /bin/ranlib -o -f /usr/ccs/bin/ranlib ) ; then \
        echo $(RANLIB) libbz2.a ; \
        $(RANLIB) libbz2.a ; \
      fi

    check: test
    test: bzip2
      @cat words1
      ./bzip2 -1  < sample1.ref > sample1.rb2
      ./bzip2 -2  < sample2.ref > sample2.rb2
      ./bzip2 -3  < sample3.ref > sample3.rb2
      ./bzip2 -d  < sample1.bz2 > sample1.tst
      ./bzip2 -d  < sample2.bz2 > sample2.tst
      ./bzip2 -ds < sample3.bz2 > sample3.tst
      cmp sample1.bz2 sample1.rb2
      cmp sample2.bz2 sample2.rb2
      cmp sample3.bz2 sample3.rb2
      cmp sample1.tst sample1.ref
      cmp sample2.tst sample2.ref
      cmp sample3.tst sample3.ref
      @cat words3

    install: bzip2 bzip2recover
      if ( test ! -d $(PREFIX)/bin ) ; then mkdir -p $(PREFIX)/bin ; fi
      if ( test ! -d $(PREFIX)/lib ) ; then mkdir -p $(PREFIX)/lib ; fi
      if ( test ! -d $(PREFIX)/man ) ; then mkdir -p $(PREFIX)/man ; fi
      if ( test ! -d $(PREFIX)/man/man1 ) ; then mkdir -p $(PREFIX)/man/man1 ; fi
      if ( test ! -d $(PREFIX)/include ) ; then mkdir -p $(PREFIX)/include ; fi
      cp -f bzip2 $(PREFIX)/bin/bzip2
      cp -f bzip2 $(PREFIX)/bin/bunzip2
      cp -f bzip2 $(PREFIX)/bin/bzcat
      cp -f bzip2recover $(PREFIX)/bin/bzip2recover
      chmod a+x $(PREFIX)/bin/bzip2
      chmod a+x $(PREFIX)/bin/bunzip2
      chmod a+x $(PREFIX)/bin/bzcat
      chmod a+x $(PREFIX)/bin/bzip2recover
      cp -f bzip2.1 $(PREFIX)/man/man1
      chmod a+r $(PREFIX)/man/man1/bzip2.1
      cp -f bzlib.h $(PREFIX)/include
      chmod a+r $(PREFIX)/include/bzlib.h
      cp -f libbz2.a $(PREFIX)/lib
      chmod a+r $(PREFIX)/lib/libbz2.a
      cp -f bzgrep $(PREFIX)/bin/bzgrep
      ln -s -f $(PREFIX)/bin/bzgrep $(PREFIX)/bin/bzegrep
      ln -s -f $(PREFIX)/bin/bzgrep $(PREFIX)/bin/bzfgrep
      chmod a+x $(PREFIX)/bin/bzgrep
      cp -f bzmore $(PREFIX)/bin/bzmore
      ln -s -f $(PREFIX)/bin/bzmore $(PREFIX)/bin/bzless
      chmod a+x $(PREFIX)/bin/bzmore
      cp -f bzdiff $(PREFIX)/bin/bzdiff
      ln -s -f $(PREFIX)/bin/bzdiff $(PREFIX)/bin/bzcmp
      chmod a+x $(PREFIX)/bin/bzdiff
      cp -f bzgrep.1 bzmore.1 bzdiff.1 $(PREFIX)/man/man1
      chmod a+r $(PREFIX)/man/man1/bzgrep.1
      chmod a+r $(PREFIX)/man/man1/bzmore.1
      chmod a+r $(PREFIX)/man/man1/bzdiff.1
      echo ".so man1/bzgrep.1" > $(PREFIX)/man/man1/bzegrep.1
      echo ".so man1/bzgrep.1" > $(PREFIX)/man/man1/bzfgrep.1
      echo ".so man1/bzmore.1" > $(PREFIX)/man/man1/bzless.1
      echo ".so man1/bzdiff.1" > $(PREFIX)/man/man1/bzcmp.1

    clean:
      rm -f *.o libbz2.a bzip2 bzip2recover \
      sample1.rb2 sample2.rb2 sample3.rb2 \
      sample1.tst sample2.tst sample3.tst

    blocksort.o: blocksort.c
      @cat words0
      $(CC) $(CFLAGS) -c blocksort.c
    huffman.o: huffman.c
      $(CC) $(CFLAGS) -c huffman.c
    crctable.o: crctable.c
      $(CC) $(CFLAGS) -c crctable.c
    randtable.o: randtable.c
      $(CC) $(CFLAGS) -c randtable.c
    compress.o: compress.c
      $(CC) $(CFLAGS) -c compress.c
    decompress.o: decompress.c
      $(CC) $(CFLAGS) -c decompress.c
    bzlib.o: bzlib.c
      $(CC) $(CFLAGS) -c bzlib.c
    bzip2.o: bzip2.c
      $(CC) $(CFLAGS) -c bzip2.c
    bzip2recover.o: bzip2recover.c
      $(CC) $(CFLAGS) -c bzip2recover.c

    distclean: clean
      rm -f manual.ps manual.html manual.pdf

    DISTNAME=bzip2-1.0.8
    dist: check manual
      rm -f $(DISTNAME)
      ln -s -f . $(DISTNAME)
      tar cvf $(DISTNAME).tar \
         $(DISTNAME)/blocksort.c \
         $(DISTNAME)/huffman.c \
         $(DISTNAME)/crctable.c \
         $(DISTNAME)/randtable.c \
         $(DISTNAME)/compress.c \
         $(DISTNAME)/decompress.c \
         $(DISTNAME)/bzlib.c \
         $(DISTNAME)/bzip2.c \
         $(DISTNAME)/bzip2recover.c \
         $(DISTNAME)/bzlib.h \
         $(DISTNAME)/bzlib_private.h \
         $(DISTNAME)/Makefile \
         $(DISTNAME)/LICENSE \
         $(DISTNAME)/bzip2.1 \
         $(DISTNAME)/bzip2.1.preformatted \
         $(DISTNAME)/bzip2.txt \
         $(DISTNAME)/words0 \
         $(DISTNAME)/words1 \
         $(DISTNAME)/words2 \
         $(DISTNAME)/words3 \
         $(DISTNAME)/sample1.ref \
         $(DISTNAME)/sample2.ref \
         $(DISTNAME)/sample3.ref \
         $(DISTNAME)/sample1.bz2 \
         $(DISTNAME)/sample2.bz2 \
         $(DISTNAME)/sample3.bz2 \
         $(DISTNAME)/dlltest.c \
         $(DISTNAME)/manual.html \
         $(DISTNAME)/manual.pdf \
         $(DISTNAME)/manual.ps \
         $(DISTNAME)/README \
         $(DISTNAME)/README.COMPILATION.PROBLEMS \
         $(DISTNAME)/README.XML.STUFF \
         $(DISTNAME)/CHANGES \
         $(DISTNAME)/libbz2.def \
         $(DISTNAME)/libbz2.dsp \
         $(DISTNAME)/dlltest.dsp \
         $(DISTNAME)/makefile.msc \
         $(DISTNAME)/unzcrash.c \
         $(DISTNAME)/spewG.c \
         $(DISTNAME)/mk251.c \
         $(DISTNAME)/bzdiff \
         $(DISTNAME)/bzdiff.1 \
         $(DISTNAME)/bzmore \
         $(DISTNAME)/bzmore.1 \
         $(DISTNAME)/bzgrep \
         $(DISTNAME)/bzgrep.1 \
         $(DISTNAME)/Makefile-libbz2_so \
         $(DISTNAME)/bz-common.xsl \
         $(DISTNAME)/bz-fo.xsl \
         $(DISTNAME)/bz-html.xsl \
         $(DISTNAME)/bzip.css \
         $(DISTNAME)/entities.xml \
         $(DISTNAME)/manual.xml \
         $(DISTNAME)/format.pl \
         $(DISTNAME)/xmlproc.sh
      gzip -v $(DISTNAME).tar

    # For rebuilding the manual from sources on my SuSE 9.1 box

    MANUAL_SRCS=  bz-common.xsl bz-fo.xsl bz-html.xsl bzip.css \
        entities.xml manual.xml

    manual: manual.html manual.ps manual.pdf

    manual.ps: $(MANUAL_SRCS)
      ./xmlproc.sh -ps manual.xml

    manual.pdf: $(MANUAL_SRCS)
      ./xmlproc.sh -pdf manual.xml

    manual.html: $(MANUAL_SRCS)
      ./xmlproc.sh -html manual.xml

## Example Bzip2 Fakefile

Featureless Make has a different design ideology that the GNU Make project. The file should, relatively speaking, be easy to read within consideration of the technical field and the complexity of the tasks at hand. The build system should not be involved with installing to the system. Installation should be a distribution level task. This does not mean that Featureless Make cannot do that. All this means is that the project is optimized to compile or build the files.

What this means is that there is not a one to one relationship between a typical `Makefile` and typical `fakefile`. This example focuses on how a `fakefile` should be designed. This is also a proof of concept design that intentionally goes out of its way to use both build systems offered by Featureless Make.

There are three files used for this example to create a `bzip2` and a `bzip2recover` binary. The first is `fakefile`.

    # fss-0005 iki-0002
    #
    # This is designed for compiling bzip2-1.0.8.
    #
    # To use this do the following:
    #   1) Download and extract bzip2-1.0.8.
    #   2) Change into the extracted bzip2-1.0.8 directory.
    #   3) Create the directory path "data/build/" within this bzip2-1.0.8 directory.
    #   4) Copy the files fakefile-bzip2, settings-bzip2, and settings-bzip2recover) into the created directory path "data/build/".
    #   5) Use the featureless make build operation to make, such as: "fake".

    settings:
      environment PATH LD_LIBRARY_PATH

    main:
      operate bzip2
      operate bzip2recover
      operate scripts
      operate tests

    bzip2:
      print
      print Building settings-bzip2.
      build settings-bzip2

    bzip2recover:
      print
      print Building settings-bzip2recover.
      build settings-bzip2recover

    scripts:
      print
      print Copying bzip2 scripts.
      touch directory build build/programs build/programs/scripts
      copy bzdiff bzgrep build/programs/scripts

    tests:
      print
      print Performing tests.

      # Conditionally setup the LD_LIBRARY_PATH to include the just compiled libraries.
      if define LD_LIBRARY_PATH
      and parameter work
        define LD_LIBRARY_PATH 'build/libraries/shared:parameter:"work:value"libraries/shared:define:"LD_LIBRARY_PATH"'

      else
      if define LD_LIBRARY_PATH
        define LD_LIBRARY_PATH 'build/libraries/shared:define:"LD_LIBRARY_PATH"'

      else
      if parameter work
        define LD_LIBRARY_PATH 'build/libraries/shared:parameter:"work:value"libraries/shared'

      # Setup the PATH to use the just compiled programs.
      define PATH programs/static:programs/shared:define:"PATH"

      # Execute the tests (redirects are not currently supported, so execute using bash).
      run cat words1
      run bash -c 'bzip2 -1  < sample1.ref > sample1.rb2'
      run bash -c 'bzip2 -2  < sample2.ref > sample2.rb2'
      run bash -c 'bzip2 -3  < sample3.ref > sample3.rb2'
      run bash -c 'bzip2 -d  < sample1.bz2 > sample1.tst'
      run bash -c 'bzip2 -d  < sample2.bz2 > sample2.tst'
      run bash -c 'bzip2 -ds < sample3.bz2 > sample3.tst'
      run cmp sample1.bz2 sample1.rb2
      run cmp sample2.bz2 sample2.rb2
      run cmp sample3.bz2 sample3.rb2
      run cmp sample1.tst sample1.ref
      run cmp sample2.tst sample2.ref
      run cmp sample3.tst sample3.ref
      run cat words3

The second is `settings-bzip2`.

    # fss-0001
    #
    # This is designed for compiling bzip2-1.0.8.
    #
    # To use this do the following:
    #   1) Download and extract bzip2-1.0.8.
    #   2) Change into the extracted bzip2-1.0.8 directory.
    #   3) Create the directory path "data/build/" within this bzip2-1.0.8 directory.
    #   4) Copy this file (settings-bzip2) into the created directory path "data/build/".
    #   5) Use the featureless make build operation to build, such as: "fake build -s settings-bzip2".

    build_name bzip2

    version_major 1
    version_minor 0
    version_micro 8
    version_file micro
    version_target major

    modes gcc clang
    modes_default gcc

    build_compiler gcc
    build_compiler-clang clang
    build_indexer ar
    build_indexer_arguments rcs
    build_language c
    build_libraries -lc
    build_sources_program blocksort.c huffman.c crctable.c randtable.c compress.c decompress.c bzlib.c bzip2.c
    build_sources_headers bzlib.h
    build_script no

    path_sources
    has_path_standard no
    preserve_path_headers no

    environment PATH LD_LIBRARY_PATH

    defines -D_FILE_OFFSET_BITS=64

    flags -z now -g -fdiagnostics-color=always -Wall -Winline -O2
    flags-clang -Wno-logical-op-parentheses
    flags_library -fPIC
    flags_program -fPIE

The third is `settings-bzip2recover`.

    # fss-0001
    #
    # This is designed for compiling bzip2-1.0.8.
    #
    # To use this do the following:
    #   1) download and extract bzip2-1.0.8.
    #   2) change into the extracted bzip2-1.0.8 directory.
    #   3) create the directory path "data/build/" within this bzip2-1.0.8 directory.
    #   4) copy this file (settings-bzip2recover) into the created directory path "data/build/".
    #   5) use the featureless make build operation to build, such as: "fake build -s settings-bzip2recover".

    build_name bzip2recover

    version_major 1
    version_minor 0
    version_micro 8
    version_file micro
    version_target major

    modes gcc clang
    modes_default gcc

    build_compiler gcc
    build_compiler-clang clang
    build_indexer ar
    build_indexer_arguments rcs
    build_language c
    build_libraries -lc
    build_sources_program bzip2recover.c
    build_script no

    path_sources
    has_path_standard no
    preserve_path_headers no

    environment PATH LD_LIBRARY_PATH

    defines -D_FILE_OFFSET_BITS=64

    flags -z now -g -fdiagnostics-color=always -Wall -Winline -O2
    flags-clang -Wno-logical-op-parentheses
    flags_library -fPIC
    flags_program -fPIE

## Example Controller Files

The Controller program has a similar design ideology to that of the Featureless Make program. A key difference is that the Controller program operates more like a service, a program, or a sysvinit alternative rather than as a build system.

An example usage would be to start the `htop` program. In this example, when the `htop` program quits, the controller program will restart after a delay for a max of 3 times. The controller command using these files would be executed using the command `controller htop-command`. Only two files are necessary to accomplish this task. The first file is the Entry file, called `entries/htop-command.entry`.

    # fss-0005

    settings:
      mode program

      control htop.socket
      control_user 0
      control_group 0
      control_mode ug+rwx,o-rwx

    main:
      start command htop

The second file is is the Rule file, called `rules/command/htop.rule`.

    # fss-000d

    settings:
      name "Run htop"

    command:
      start htop

      rerun start success delay 3000 max 3