Difference between revisions of "ARK2/Frontend"

Revision as of 11:42, 16 April 2018

Overview

ARK2 has imposed a clear separation between the backend and frontend, allowing the frontend to operate completely independently. ARK2 provides a number of standard frontend implementations, but sites can adapt or implement their own. This also supports multi-tenant mode where a single ARK2 install can be configured to run multiple ARK2 sites using different frontends.

Two options exist for building frontends:

• Integrated frontends using the ARK2 PHP API via TWIG templates and Views defined in the config database
• Standalone frontends using calls to the ARK2 REST API

The integrated frontends that ship with ARK2 are implemented in Bootstrap, jQuery, and Twig, the most popular and well-supported frontend libraries. Frontends can be cloned and modified for a specific site, allowing for easier customisation of ARK's appearance by third party designers. Standalone frontends could be implemented in any other technologies such as React and Vue.

The integrated frontends shipped with ARK2 are:

• admin - A minimal Web Admin front-end for configuring an API-only server
• ark2 - The default front-end supporting the full ARK2 feature set
• bootstrap3 - A minimal Bootstrap 3 frontend designed as a base for custom frontends
• bootstrap4 - A minimal Bootstrap 4 frontend designed as a base for custom frontends
• flat - A simple, non-js, read-only front-end designed for scraping, e.g. for static archiving or web-bots

The integrated frontends require a build process to assemble the required frontend resources into a distributable form. This page documents the process required to do so.

The resources built into a frontend can be obtained from three sources:

• Vendor resources: Third-party libraries from outside ARK2, such as Bootstrap, jQuery, and FontAwesome
• Core resources: Standard resources provided by ARK2 for interacting with ARK2 features, e.g. common widgets, form layouts, etc
• Custom resources: Custom resources developed for a particular frontend

The build process uses tooling to combine these resources into a single source package that is able to be distributed.

Source

Packaged frontend resources are saved in the src directory by Namespace and Name:

 src/<namespace>/frontend/<frontend>

For example, the main ark2 frontend in the ARK name space is stored in:

 src/ARK/frontend/ark2

Within each frontend source folder, the source is organised by general type:

 |- <frontend>/
   |- assets/
     |- fonts/
     |- images/
     |- scripts/
     |- styles/
   |- bin/
     |- console
   |- config/
     |- credentials.json
     |- database.json
     |- site.json
   |- public/
     |- .htaccess
     |- index.php
   |- templates/
   |- translations/

The following source types generally use the default source and remain unchanged:

• bin - Any executables used by the frontend, usually just the site console
• config - The default config files for the frontend, the main site.json file, the database.json file, and the credentials.json file
• public - The default contents of the public webroot folder, including the index.php file

The following source types are usually modified for a custom frontend and rebuilt using the build tooling when changed:

• assets - The public assets for the frontend that get included under the public webroot, i.e. fonts, images, scripts and stylesheets
• templates - The twig templates used to render the html
• translations - Vendor provided translation files

Customising the scripts and stylesheets requires various tooling which is covered in the next section.

Build Tooling

Build tooling for frontend resources is required for a number of reasons:

• Bootstrap can be customised most easily by changing variables used in the Sass templates, which then requires a build step to compile them into CSS
• Production deployment is more efficient if CSS and JS is stripped, merged and minified, while development is easier if a map is generated for the original code
• Resources from multiple sources need to be combined into a single package
• Multiple versions and combinations of resources may be required for performance reasons
• NPM library management downloads the entire package, not just the resources required, an extra step is required to copy just the required resources into the source folder
• All the steps required for packaging and release management can be automated, e.g. clean, compile, tag, package, etc
• The build tooling for the default ARK bootstrap and twig theme can be generalised to allow clients to build and deploy their own customised themes with minimal effort

The build tooling works as follows:

• All build tooling and resources are isolated in the /build folder so it can be excluded from any release packages or production deployments
• Nothing in the /build folder may be depended on by any code outside the /build folder or required for running ARK itself
• Node, npm, Gulp and Symfony Console are used to run the tooling
• NPM is used to manage the Node packages used which are installed into /build/node_packages
• Node packages are used both for the build tools themselves and for vendor libraries providing resources
• Gulp is used to ensure the build scripts run cross-platform
• Symfony Console is used to manage and run the build tasks
• Running tasks will only work inside the /build folder, trying to run outside the build folder will fail
• The resources required to be built for a frontend are defined in a manifest file
• The built frontend resources are then installed into the frontend source directory
• Later, each configured site in the ARK2 install links to the frontend it wishes to use

Build

The front end build is managed by the Build Console. The Build console is in the 'build/' folder and is used to build/rebuild/install a front-end if required. To run the console you must be in the build folder:

 cd build
 ./build

Running the console without a chosen command will show the list of available commands:

ARK Build Console 1.9.80

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands:
  help                Displays help for a command
  list                Lists commands
 cache
  cache:clear         Clear the system cache
 database
  database:reverse    Reverse engineer an existing database as DoctrineXML
 frontend
  frontend:all        Build an ARK frontend. (Args: frontend)
  frontend:assets     Build the frontend assets (Fonts, Images, Scripts, Styles). (Args: frontend)
  frontend:base       Build the frontend base (Bin, Config, Templates, Translations, Web). (Args: frontend)
  frontend:create     Create a new ARK frontend (Args: namespace, frontend).
  frontend:scripts    Build the frontend scripts (JavaScript). (Args: frontend)
  frontend:styles     Build the frontend styles (CSS and SASS). (Args: frontend)
  frontend:templates  Build the frontend templates (Twig). (Args: frontend)

The first time you need to build/rebuild a frontend, you need to install the required Node packages. You will need to have installed Node and NPM on your system first.

 npm install

To update an existing installation, especially after an ARK upgrade:

 npm update

To check the status of your Node environment (including available updates), run:

 npm doctor

To build a frontend run the frontend command with the name of the frontend:

 ./build frontend:all <frontend>

This will install the built front end into 'src/<namespace>/frontend/<frontend>'.

You can also choose to build just one type of resource to save time, e.g. if you only changed a template and don't want to wait for the js to be rebuilt as well:

 ./build frontend:styles <frontend>
 ./build frontend:scripts <frontend>
 ./build frontend:templates <frontend>
 ./build frontend:assets <frontend>
 ./build frontend:base <frontend>

The /build folder is organised by resource source as follows:

|- build/
  |- core/
  |- frontends/
    |- <frontend1>/
    |- <frontend2>/
    |- <frontend3>/
  |- node_modules/
  |- .jsbeautifyrc
  |- .jshintrc
  |- build
  |- gulpfile.js
  |- packages.json

• The /node_modules folder holds the vendor resources managed by NPM as defined in the packages.json file
• The/ core folder holds the core resources provided by ark2
• The /frontends folder holds the custom resources for the various integrated frontends
• The /gulpfile.js defines the Gulp build scripts
• The /build console runs the build scripts
• The /.jsbeautifyrc and /.jshintrc files define the JavaScript coding standards used in ARK2

Within the /core folder and each custom frontend folder, the resources provided are organised in folders by resource source type:

 |- <frontend>/
   |- bin/
   |- config/
   |- fonts/
   |- images/
   |- js/
   |- public/
   |- scss/
   |- twig/
   |- xliff/

Manifest

The combination of resources that are built into a frontend are defined by the build manifest.json file in the root of each frontend folder, e.g. /build/frontends/ark2/manifest.json.

{
    "frontend": "ark2",
    "namespace": "ARK",
    "options": {},
    "assets": {
        "fonts": {},
        "images": {},
        "scripts": {},
        "styles": {}
    },
    "bin": {},
    "config": {},
    "templates": {},
    "translations": {},
    "public": {}
}

Here you see the root level of the manifest defines the same resource classes as is used to organise the frontend source folder, i.e. these sections define the resources that will be packaged together to be installed into the source folder.

There are also three extra root level keys:

• frontend - the name of the frontend used in the source path
• namespace - the namespace of the frontend used in the source path
• options - various options to be used in the build tooling, keyed by the Gulp module that uses them

The default build options are as follows:

"options": {
    "uglify": {
        "compress": false
    },
    "sass": {
        "outputStyle": "compressed",
        "precision": 8,
        "includePaths": [
            "node_modules/bootstrap-sass/assets/stylesheets"
        ]
    },
    "sourcemaps": {
        "loadMaps": true
    },
    "autoprefixer": {
        "browsers": [
            "Android 2.3",
            "Android >= 4",
            "Chrome >= 20",
            "Firefox >= 24",
            "Explorer >= 8",
            "iOS >= 6",
            "Opera >= 12",
            "Safari >= 6"
        ]
    }
},

Not that the autoprefixer settings are those required by Bootstrap and should not be altered.

Within each resource class, you must define what resources are to be included from the three resource sources: vendor, core and custom.

For most resource classes this defines what files to copy into the source folder, either by individual file names or (more usually for core and custom) all files in a folder:

"templates": {
    "vendor": [
        "symfony-collection/jquery.collection.html.twig"
    ],
    "core": [
        "twig/**/*",
        "twig/**/.*"
    ],
    "custom": [
        "twig/**/*",
        "twig/**/.*"
    ]
},

When copying files in this way, the order used is vendor, core, then custom, so files in core will override files in vendor and files in custom will override files in core. The vendor file paths are relative to /build/node_modules, the core files are paths relative to /build/core, and the custom files are paths relative to /build/frontends/<frontend>.

The scripts and styles groups work differently however as they must call the SASS and JavaScript build tools to compile the destination source code.

The scripts class defines a set of build targets, that is named minified javascript files to be compiled, and the javascript files that are merged to create that file. For example, to build script files ark-default.min.js and ark-map.min.js you could define the following:

"scripts": {
    "ark-default": {
        "vendor": [
            "jquery/dist/jquery.js",
            "bootstrap-sass/assets/javascripts/bootstrap.js"
        ],
        "core": [
            "js/core-ready.js"
        ],
        "custom": [
            "js/frontend-ready.js",
        ]
    },
    "ark-map": {
        "vendor": [
            "proj4/dist/proj4.js",
            "openlayers/dist/ol.js"
        ],
        "core": [],
        "custom": [
            "js/map-ready.js"
        ]
    }
},

The javascript files are merged in the order defined in the groups, so it is recommended to explicitly list the files so that the correct order of dependencies can be defined. Sourcemap files will be created if specified in the options.

The scripts class defines a set of build targets, that is named minified CSS stylesheet files to be compiled, and the list of Gulp build streams required to built by the SASS compiler before being combined into that target file. For example, to build style files ark-default.min.css and ark-map.min.css you could define the following:

"styles": {
    "ark-default": [{
            "stream": "bootstrap",
            "vendor": [],
            "core": [],
            "custom": [
                "scss/vendor.scss"
            ]
        },
        {
            "stream": "main",
            "vendor": [
                "font-awesome/css/font-awesome.css"
            ],
            "core": [],
            "custom": [
                "scss/default.scss"
            ]
        }
    ],
    "ark-map": [{
        "stream": "main",
        "vendor": [
            "openlayers/dist/ol.css"
        ],
        "core": [],
        "custom": [
            "scss/map.scss"
        ]
    }]
}

The defining of streams within each target allows for the mixing of SASS and CSS source in a defined order so the cascading of stylesheets is applied in the correct order

Linking

The front-end is built from the source resource files in the 'build/' folder and installed in the 'src/' folder. The individual site folders then symlink to the required front-end in 'src/'.

To clone the build resources for a new front end from another front-end, run the frontend:create command. You must provide a name for your new front end, and the source code Namespace it will use, e.g. ARK, Wibble, or Foo. Only use the ARK namespace if the front end is intended for the main ARK project. This Namespace will be remembered by the system.

 ./build frontend:create <namespace> <frontend>

To use a front end for a site, run the System Console in the '<install>/bin' folder.

To create an entirely new site, use the create command. This will create a new database, site folder, config and web subfolders, and link/copy the chosen front-end.

 cd ../bin
 ./sysadmin database:server:add <server>
 ./sysadmin site:create <site> <frontend=core>

You will be asked if you want to link or copy the front-end. It is recommended to use the default link option as you will not need to update the site folder when the front-end changes. Only copy the front-end if you want to make small changes to the front-end without having to build a new namespaced front-end. This however is discouraged, you should build a namespaced front-end instead.

To enable a front-end other than core, you will need to edit the '<site>/config/site.json' file to set the name of the front-end. [TODO: Write a command for this!]

To add another front-end to a site, or to refresh a copied front-end:

 site:frontend <site> <frontend>

Useful References

• http://www.helloerik.com/the-subtle-magic-behind-why-the-bootstrap-3-grid-works