Difference between revisions of "ARK2/Frontend"

Latest revision as of 14:34, 16 April 2018

Overview

ARK2 has imposed a clear separation between the backend and frontend, allowing the frontend to operate completely independently. ARK2 provides a number of standard frontend implementations, but sites can adapt or implement their own. This also supports multi-tenant mode where a single ARK2 install can be configured to run multiple ARK2 sites using different frontends.

Two options exist for building frontends:

• Integrated frontends using the ARK2 PHP API via TWIG templates and Views defined in the config database
• Standalone frontends using calls to the ARK2 REST API

The integrated frontends that ship with ARK2 are implemented in Bootstrap, jQuery, and Twig, the most popular and well-supported frontend libraries. Frontends can be cloned and modified for a specific site, allowing for easier customisation of ARK's appearance by third party designers. Standalone frontends could be implemented in any other technologies such as React and Vue.

The integrated frontends shipped with ARK2 are:

• admin - A minimal Web Admin front-end for configuring an API-only server
• ark2 - The default front-end supporting the full ARK2 feature set
• bootstrap3 - A minimal Bootstrap 3 frontend designed as a base for custom frontends
• bootstrap4 - A minimal Bootstrap 4 frontend designed as a base for custom frontends
• flat - A simple, non-js, read-only front-end designed for scraping, e.g. for static archiving or web-bots

The integrated frontends require a build process to assemble the required frontend resources into a distributable form. This page documents the process required to do so.

The resources built into a frontend can be obtained from three sources:

• Vendor resources: Third-party libraries from outside ARK2, such as Bootstrap, jQuery, and FontAwesome
• Core resources: Standard resources provided by ARK2 for interacting with ARK2 features, e.g. common widgets, form layouts, etc
• Custom resources: Custom resources developed for a particular frontend

The build process uses tooling to combine these resources into a single source package that is able to be distributed.

Source

Packaged frontend resources are saved in the src directory by Namespace and Name:

 src/<namespace>/frontend/<frontend>

For example, the main ark2 frontend in the ARK name space is stored in:

 src/ARK/frontend/ark2

Within each frontend source folder, the source is organised by general type:

 |- <frontend>/
   |- assets/
     |- fonts/
     |- images/
     |- scripts/
     |- styles/
   |- bin/
     |- console
   |- config/
     |- credentials.json
     |- database.json
     |- site.json
   |- public/
     |- .htaccess
     |- index.php
   |- templates/
   |- translations/

The following source types generally use the default source and remain unchanged:

• bin - Any executables used by the frontend, usually just the site console
• config - The default config files for the frontend, the main site.json file, the database.json file, and the credentials.json file
• public - The default contents of the public webroot folder, including the index.php file

The following source types are usually modified for a custom frontend and rebuilt using the build tooling when changed:

• assets - The public assets for the frontend that get included under the public webroot, i.e. fonts, images, scripts and stylesheets
• templates - The twig templates used to render the html
• translations - Vendor provided translation files

Customising the scripts and stylesheets requires various tooling which is covered in the next section.

Build Tooling

Build tooling for frontend resources is required for a number of reasons:

• Bootstrap can be customised most easily by changing variables used in the Sass templates, which then requires a build step to compile them into CSS
• Production deployment is more efficient if CSS and JS is stripped, merged and minified, while development is easier if a map is generated for the original code
• Resources from multiple sources need to be combined into a single package
• Multiple versions and combinations of resources may be required for performance reasons
• NPM library management downloads the entire package, not just the resources required, an extra step is required to copy just the required resources into the source folder
• All the steps required for packaging and release management can be automated, e.g. clean, compile, tag, package, etc
• The build tooling for the default ARK bootstrap and twig theme can be generalised to allow clients to build and deploy their own customised themes with minimal effort

The build tooling works as follows:

• All build tooling and resources are isolated in the /build folder so it can be excluded from any release packages or production deployments
• Nothing in the /build folder may be depended on by any code outside the /build folder or required for running ARK itself
• Node, npm, Gulp and Symfony Console are used to run the tooling
• NPM is used to manage the Node packages used which are installed into /build/node_packages
• Node packages are used both for the build tools themselves and for vendor libraries providing resources
• Gulp is used to ensure the build scripts run cross-platform
• Symfony Console is used to manage and run the build tasks
• Running tasks will only work inside the /build folder, trying to run outside the build folder will fail
• The resources required to be built for a frontend are defined in a manifest file
• The built frontend resources are then installed into the frontend source directory
• Later, each configured site in the ARK2 install links to the frontend it wishes to use

Build

The front end build is managed by the Build Console. The Build console is in the 'build/' folder and is used to build/rebuild/install a front-end if required. To run the console you must be in the build folder:

 cd build
 ./build

Running the console without a chosen command will show the list of available commands:

ARK Build Console 1.9.80

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands:
  help                Displays help for a command
  list                Lists commands
 cache
  cache:clear         Clear the system cache
 database
  database:reverse    Reverse engineer an existing database as DoctrineXML
 frontend
  frontend:all        Build an ARK frontend. (Args: frontend)
  frontend:assets     Build the frontend assets (Fonts, Images, Scripts, Styles). (Args: frontend)
  frontend:base       Build the frontend base (Bin, Config, Templates, Translations, Web). (Args: frontend)
  frontend:create     Create a new ARK frontend (Args: namespace, frontend).
  frontend:scripts    Build the frontend scripts (JavaScript). (Args: frontend)
  frontend:styles     Build the frontend styles (CSS and SASS). (Args: frontend)
  frontend:templates  Build the frontend templates (Twig). (Args: frontend)

The first time you need to build/rebuild a frontend, you need to install the required Node packages. You will need to have installed Node and NPM on your system first.

 npm install

To update an existing installation, especially after an ARK upgrade:

 npm update

To check the status of your Node environment (including available updates), run:

 npm doctor

To install a new node package to use in the frontend, run:

 npm install <package>

To build a frontend run the frontend command with the name of the frontend:

 ./build frontend:all <frontend>

This will install the built front end into 'src/<namespace>/frontend/<frontend>'.

You can also choose to build just one type of resource to save time, e.g. if you only changed a template and don't want to wait for the js to be rebuilt as well:

 ./build frontend:styles <frontend>
 ./build frontend:scripts <frontend>
 ./build frontend:templates <frontend>
 ./build frontend:assets <frontend>
 ./build frontend:base <frontend>

The /build folder is organised by resource source as follows:

|- build/
  |- core/
  |- frontends/
    |- <frontend1>/
    |- <frontend2>/
    |- <frontend3>/
  |- node_modules/
  |- .jsbeautifyrc
  |- .jshintrc
  |- build
  |- gulpfile.js
  |- packages.json

• The /node_modules folder holds the vendor resources managed by NPM as defined in the packages.json file
• The/ core folder holds the core resources provided by ark2
• The /frontends folder holds the custom resources for the various integrated frontends
• The /gulpfile.js defines the Gulp build scripts
• The /build console runs the build scripts
• The /.jsbeautifyrc and /.jshintrc files define the JavaScript coding standards used in ARK2

Within the /core folder and each custom frontend folder, the resources provided are organised in folders by resource source type:

 |- <frontend>/
   |- bin/
   |- config/
   |- fonts/
   |- images/
   |- js/
   |- public/
   |- scss/
   |- twig/
   |- xliff/

Manifest

The combination of resources that are built into a frontend are defined by the build manifest.json file in the root of each frontend folder, e.g. /build/frontends/ark2/manifest.json.

{
    "frontend": "ark2",
    "namespace": "ARK",
    "options": {},
    "assets": {
        "fonts": {},
        "images": {},
        "scripts": {},
        "styles": {}
    },
    "bin": {},
    "config": {},
    "templates": {},
    "translations": {},
    "public": {}
}

Here you see the root level of the manifest defines the same resource classes as is used to organise the frontend source folder, i.e. these sections define the resources that will be packaged together to be installed into the source folder.

There are also three extra root level keys:

• frontend - the name of the frontend used in the source path
• namespace - the namespace of the frontend used in the source path
• options - various options to be used in the build tooling, keyed by the Gulp module that uses them

The default build options are as follows:

"options": {
    "uglify": {
        "compress": false
    },
    "sass": {
        "outputStyle": "compressed",
        "precision": 8,
        "includePaths": [
            "node_modules/bootstrap-sass/assets/stylesheets"
        ]
    },
    "sourcemaps": {
        "loadMaps": true
    },
    "autoprefixer": {
        "browsers": [
            "Android 2.3",
            "Android >= 4",
            "Chrome >= 20",
            "Firefox >= 24",
            "Explorer >= 8",
            "iOS >= 6",
            "Opera >= 12",
            "Safari >= 6"
        ]
    }
},

Not that the autoprefixer settings are those required by Bootstrap and should not be altered.

Within each resource class, you must define what resources are to be included from the three resource sources: vendor, core and custom.

For most resource classes this defines what files to copy into the source folder, either by individual file names or (more usually for core and custom) all files in a folder:

"templates": {
    "vendor": [
        "symfony-collection/jquery.collection.html.twig"
    ],
    "core": [
        "twig/**/*",
        "twig/**/.*"
    ],
    "custom": [
        "twig/**/*",
        "twig/**/.*"
    ]
},

When copying files in this way, the order used is vendor, core, then custom, so files in core will override files in vendor and files in custom will override files in core. The vendor file paths are relative to /build/node_modules, the core files are paths relative to /build/core, and the custom files are paths relative to /build/frontends/<frontend>.

The scripts and styles groups work differently however as they must call the SASS and JavaScript build tools to compile the destination source code.

The scripts class defines a set of build targets, that is named minified javascript files to be compiled, and the javascript files that are merged to create that file. For example, to build script files ark-default.min.js and ark-map.min.js you could define the following:

"scripts": {
    "ark-default": {
        "vendor": [
            "jquery/dist/jquery.js",
            "bootstrap-sass/assets/javascripts/bootstrap.js"
        ],
        "core": [
            "js/core-ready.js"
        ],
        "custom": [
            "js/frontend-ready.js",
        ]
    },
    "ark-map": {
        "vendor": [
            "proj4/dist/proj4.js",
            "openlayers/dist/ol.js"
        ],
        "core": [],
        "custom": [
            "js/map-ready.js"
        ]
    }
},

The javascript files are merged in the order defined in the groups, so it is recommended to explicitly list the files so that the correct order of dependencies can be defined. Sourcemap files will be created if specified in the options.

The scripts class defines a set of build targets, that is named minified CSS stylesheet files to be compiled, and the list of Gulp build streams required to built by the SASS compiler before being combined into that target file. For example, to build style files ark-default.min.css and ark-map.min.css you could define the following:

"styles": {
    "ark-default": [{
            "stream": "bootstrap",
            "vendor": [],
            "core": [],
            "custom": [
                "scss/vendor.scss"
            ]
        },
        {
            "stream": "main",
            "vendor": [
                "font-awesome/css/font-awesome.css"
            ],
            "core": [],
            "custom": [
                "scss/default.scss"
            ]
        }
    ],
    "ark-map": [{
        "stream": "main",
        "vendor": [
            "openlayers/dist/ol.css"
        ],
        "core": [],
        "custom": [
            "scss/map.scss"
        ]
    }]
}

The defining of streams within each target allows for the mixing of SASS and CSS source in a defined order so the cascading of stylesheets is applied in the correct order

Templates

Twig template resources are organised using a convention to make dynamic mixing of templates to create many different Views possible.

The twig folder for a frontend is organised as follows:

|- twig/
  |- blocks/
  |- errors/
  |- framework/
  |- layouts/
  |- pages/
  |- profiler/
  |- user/
    |- emails/
    |- layouts/

• blocks - Small chunks of twig code designed to be reused and repeated using Twig Block inheritance, e.g. form fields, widgets, etc.
• layouts - Arrangements of blocks into generic or task specific layouts, e.g. grids, tables, custom content views, etc.
• pages - Definitions of the sets of standard blocks to use when building the HTML document for a particular page
• framework - Definition of the standard HTML document and default page blocks
• errors - Standard error pages, i.e. 404, 500, etc
• user - Required layouts for the user security system, e.g. user verification emails, login forms, etc.
• profiler - Custom templates to be used in the Symfony profiler

The blocks used fall into three general groups:

• Standard page blocks - blocks that define or override standard page elements such as scripts, navbars, etc, which are saved in separate block files.
• Form blocks - blocks that define standard Symfony Form elements such as Textarea, Select, Radio, etc, which are saved in a single blocks/forms.html.twig file
• Custom widget blocks - blocks that define custom widgets that do not use Symfony Forms and are saved in separate block files

The standard page blocks are:

• head - defines the document <head> element
• styles - stylesheets to be included at the top of the document <head> element
• body - defines the document <body> element, usually how the navbar, sidebar, content and footer blocks are arranged in the body
• navbar - defines the navbar or header at the top of the document body
• sidebar - defines the sidebar or menu at the side of the document body
• content - defines the main content of the document body
• footer - defines the footer at the bottom of the document body
• scripts - scripts to be included at the bottom of the document <body> element
• link - defines the link element used in the navbar/sidebar/footer
• flash - defines the page flash element

The standard ARK2 page view is built in Twig as follows:

• framework/doc.html.twig - Skeleton of the HTML5 document, includes undefined blocks for styles, head, body and scripts
• framework/page.html.twig - Extends doc.html.twig and defines the full default set of blocks to use for styles, head, body, and scripts, includes other default blocks used by those blocks such as navbar, but excludes the content block to be used
• pages/<page>.html.twig - Extends framework/page.html.twig to set the content block to be used by a page, and overrides any standard blocks with custom blocks to be used, e.g. override the standard styles or navbar blocks on this page. This is the root template rendered by a page view.

Scripts

The use of Javascript in ARK2 is not very well defined or organised as yet beyond the use of jQuery and various Bootstrap-based widgets.

Core Javascript resources in /build/core/js:

• alert.js - Utilies for working with the ARK2/Bootstrap alerts
• core-ready.js - Initialise the core javascript on document ready
• form.js - Various form related tools, such as the AJAX form mappper
• form-ready.js - Initialise the form javascript on document ready
• location.js - An OpenLayers map picker widget as configured in the ark_map table.
• map-ol4.js - An OpenLayers map widget as configured in the ark_map table.
• map-ready.js - Initialise the map widget on document ready
• pageedit.js - Static page editing using Summernote Editor
• routing.js - The Routing module to use to generate paths from Route IDs.
• utilities.js - Some utility routines, such as debounce.

Common vendor libraries used via NPM:

• jquery
• jquery-form
• moment
• bazinga-translator
• select2
• bootstrap
• bootbox
• bootstrap-datetime-picker
• bootstrap-fileinput
• bootstrap-table

Styles

Styles use a combination of SASS and CSS as required with Bootstrap as the chosen ui style.

Useful References

• https://symfony.com/doc/3.4/form/form_customization.html
• https://symfony.com/doc/3.4/reference/forms/twig_reference.html
• http://www.helloerik.com/the-subtle-magic-behind-why-the-bootstrap-3-grid-works