Difference between revisions of "ARK2/Frontend"

From ARK
Jump to: navigation, search
(Build Tooling)
(Scripts)
 
(52 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Frontend =
+
== Overview ==
  
ARK2 has imposed a clear separation between the backend and front-end, allowing the front-end to operate completely independently. ARK2 provides a number of standard front-end implementations, but sites can adapt or implement their own.
+
ARK2 has imposed a clear separation between the backend and frontend, allowing the frontend to operate completely independently. ARK2 provides a number of standard frontend implementations, but sites can adapt or implement their own. This also supports multi-tenant mode where a single ARK2 install can be configured to run multiple ARK2 sites using different frontends.
  
The standard frontends are implemented in [http://getbootstrap.com/ Bootstrap], [https://jquery.com/ jQuery], and [http://twig.sensiolabs.org/ Twig], the most popular and well-supported front-end ui component and template systems. This allows for easier customisation of ARK's appearance by third party designers.
+
Two options exist for building frontends:
 +
* Integrated frontends using the ARK2 PHP API via TWIG templates and Views defined in the config database
 +
* Standalone frontends using calls to the ARK2 REST API
  
The standard front-ends shipped with ARK2 are:
+
The integrated frontends that ship with ARK2 are implemented in [http://getbootstrap.com/ Bootstrap], [https://jquery.com/ jQuery], and [http://twig.sensiolabs.org/ Twig], the most popular and well-supported frontend libraries. Frontends can be cloned and modified for a specific site, allowing for easier customisation of ARK's appearance by third party designers. Standalone frontends could be implemented in any other technologies such as React and Vue.
* api - The absolute minimal front-end required to run an API-only server, all admin is performed via the admin console.
+
 
 +
The integrated frontends shipped with ARK2 are:
 
* admin - A minimal Web Admin front-end for configuring an API-only server
 
* admin - A minimal Web Admin front-end for configuring an API-only server
 +
* ark2 - The default front-end supporting the full ARK2 feature set
 +
* bootstrap3 - A minimal Bootstrap 3 frontend designed as a base for custom frontends
 +
* bootstrap4 - A minimal Bootstrap 4 frontend designed as a base for custom frontends
 
* flat - A simple, non-js, read-only front-end designed for scraping, e.g. for static archiving or web-bots
 
* flat - A simple, non-js, read-only front-end designed for scraping, e.g. for static archiving or web-bots
* core - The default front-end supporting the full ARK2 feature set
 
  
== Build ==
+
The integrated frontends require a build process to assemble the required frontend resources into a distributable form. This page documents the process required to do so.
 +
 
 +
The resources built into a frontend can be obtained from three sources:
 +
* Vendor resources: Third-party libraries from outside ARK2, such as Bootstrap, jQuery, and FontAwesome
 +
* Core resources: Standard resources provided by ARK2 for interacting with ARK2 features, e.g. common widgets, form layouts, etc
 +
* Custom resources: Custom resources developed for a particular frontend
 +
 
 +
The build process uses tooling to combine these resources into a single source package that is able to be distributed.
 +
 
 +
== Source ==
 +
 
 +
Packaged frontend resources are saved in the src directory by Namespace and Name:
 +
 
 +
  src/<namespace>/frontend/<frontend>
 +
 
 +
For example, the main ark2 frontend in the ARK name space is stored in:
 +
 
 +
  src/ARK/frontend/ark2
 +
 
 +
Within each frontend source folder, the source is organised by general type:
 +
 
 +
  |- <frontend>/
 +
    |- assets/
 +
      |- fonts/
 +
      |- images/
 +
      |- scripts/
 +
      |- styles/
 +
    |- bin/
 +
      |- console
 +
    |- config/
 +
      |- credentials.json
 +
      |- database.json
 +
      |- site.json
 +
    |- public/
 +
      |- .htaccess
 +
      |- index.php
 +
    |- templates/
 +
    |- translations/
 +
 
 +
The following source types generally use the default source and remain unchanged:
 +
* bin - Any executables used by the frontend, usually just the site console
 +
* config - The default config files for the frontend, the main site.json file, the database.json file, and the credentials.json file
 +
* public - The default contents of the public webroot folder, including the index.php file
 +
 
 +
The following source types are usually modified for a custom frontend and rebuilt using the build tooling when changed:
 +
* assets - The public assets for the frontend that get included under the public webroot, i.e. fonts, images, scripts and stylesheets
 +
* templates - The twig templates used to render the html
 +
* translations - Vendor provided translation files
 +
 
 +
Customising the scripts and stylesheets requires various tooling which is covered in the next section.
 +
 
 +
== Build Tooling ==
 +
 
 +
Build tooling for frontend resources is required for a number of reasons:
 +
* Bootstrap can be customised most easily by changing variables used in the Sass templates, which then requires a build step to compile them into CSS
 +
* Production deployment is more efficient if CSS and JS is stripped, merged and minified, while development is easier if a map is generated for the original code
 +
* Resources from multiple sources need to be combined into a single package
 +
* Multiple versions and combinations of resources may be required for performance reasons
 +
* NPM library management downloads the entire package, not just the resources required, an extra step is required to copy just the required resources into the source folder
 +
* All the steps required for packaging and release management can be automated, e.g. clean, compile, tag, package, etc
 +
* The build tooling for the default ARK bootstrap and twig theme can be generalised to allow clients to build and deploy their own customised themes with minimal effort
  
The front-end is built from the source assets files in the 'build/' folder and installed in the 'src/' folder. The individual site folders then symlink to the required front-end in 'src/', or copy if they wish to edit in place. (See the [[ARK2/Frontend#File Layout|FileLayout]] section below for more on this).
+
The build tooling works as follows:
 +
* All build tooling and resources are isolated in the /build folder so it can be excluded from any release packages or production deployments
 +
* Nothing in the /build folder may be depended on by any code outside the /build folder or required for running ARK itself
 +
* [https://nodejs.org/en/ Node], [https://www.npmjs.com/ npm], [http://gulpjs.com/ Gulp] and Symfony Console are used to run the tooling
 +
* NPM is used to manage the Node packages used which are installed into /build/node_packages
 +
* Node packages are used both for the build tools themselves and for vendor libraries providing resources
 +
* Gulp is used to ensure the build scripts run cross-platform
 +
* Symfony Console is used to manage and run the build tasks
 +
* Running tasks will only work inside the /build folder, trying to run outside the build folder will fail
 +
* The resources required to be built for a frontend are defined in a manifest file
 +
* The built frontend resources are then installed into the frontend source directory
 +
* Later, each configured site in the ARK2 install links to the frontend it wishes to use
  
The front end build and install is managed by the [[ARK2/Console|Build and System Consoles]]. The Build console is in the 'build/' folder and is used to build/rebuild/install a front-end if required. The System Console is in the 'build/' folder and is used to link the installed front end to a site.
+
== Build ==
  
To build a front end you first need to install the build environment (see [[ARK2/Technical#Environment|Environment]] for installing the prerequisites):
+
The front end build is managed by the [[ARK2/Console|Build Console]]. The Build console is in the 'build/' folder and is used to build/rebuild/install a front-end if required. To run the console you must be in the build folder:
  
 
   cd build
 
   cd build
   npm update
+
   ./build
  
You may occasionally need to re-run this command to update the build environment, especially after an ARK upgrade.
+
Running the console without a chosen command will show the list of available commands:
  
To see details of the build environment (including available updates), run:
+
<pre>
 +
ARK Build Console 1.9.80
  
   ./build env:status
+
Usage:
 +
   command [options] [arguments]
  
To clone the build assets for a new front end from the Core front-end, run the create command. You must provide a name for your new front end, and the source code Namespace it will use, e.g. ARK, Wibble, or Foo. Only use the ARK namespace if the front end is intended for the main ARK project. This Namespace will be remembered by the system.
+
Options:
 +
  -h, --help            Display this help message
 +
  -q, --quiet          Do not output any message
 +
  -V, --version        Display this application version
 +
      --ansi            Force ANSI output
 +
      --no-ansi        Disable ANSI output
 +
  -n, --no-interaction  Do not ask any interactive question
 +
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
  
   ./build frontend:create <namespace> <frontend>
+
Available commands:
 +
  help                Displays help for a command
 +
  list                Lists commands
 +
cache
 +
  cache:clear        Clear the system cache
 +
database
 +
  database:reverse    Reverse engineer an existing database as DoctrineXML
 +
frontend
 +
   frontend:all        Build an ARK frontend. (Args: frontend)
 +
  frontend:assets    Build the frontend assets (Fonts, Images, Scripts, Styles). (Args: frontend)
 +
  frontend:base      Build the frontend base (Bin, Config, Templates, Translations, Web). (Args: frontend)
 +
  frontend:create     Create a new ARK frontend (Args: namespace, frontend).
 +
  frontend:scripts    Build the frontend scripts (JavaScript). (Args: frontend)
 +
  frontend:styles    Build the frontend styles (CSS and SASS). (Args: frontend)
 +
  frontend:templates  Build the frontend templates (Twig). (Args: frontend)
 +
</pre>
  
To build and install a front-end run the front-end command. You can supply a front end name to install or default to the 'core' front end.
+
The first time you need to build/rebuild a frontend, you need to install the required Node packages. You will need to have installed Node and NPM on your system first.
  
   ./console frontend:all <frontend=core>
+
   npm install
  
This will install the built front end into 'src/<namespace>/frontend/<frontend>'.
+
To update an existing installation, especially after an ARK upgrade:
  
You can also choose to build just one type of asset to save time, e.g. if you only changed a template and don't want to wait for the js to be rebuilt as well:
+
  npm update
  
  ./build frontend:css <frontend=core>
+
To check the status of your Node environment (including available updates), run:
  ./build frontend:js <frontend=core>
 
  ./build frontend:twig <frontend=core>
 
  
To use a front end for a site, run the System Console in the '<install>/bin' folder.
+
  npm doctor
  
To create an entirely new site, use the create command. This will create a new database, site folder, config and web subfolders, and link/copy the chosen front-end.
+
To install a new node package to use in the frontend, run:
  
   cd ../bin
+
   npm install <package>
  ./sysadmin database:server:add <server>
 
  ./sysadmin site:create <site> <frontend=core>
 
  
You will be asked if you want to link or copy the front-end. It is recommended to use the default link option as you will not need to update the site folder when the front-end changes. Only copy the front-end if you want to make small changes to the front-end without having to build a new namespaced front-end. This however is discouraged, you should build a namespaced front-end instead.
+
To build a frontend run the frontend command with the name of the frontend:
  
To enable a front-end other than core, you will need to edit the '<site>/config/site.json' file to set the name of the front-end. [TODO: Write a command for this!]
+
  ./build frontend:all <frontend>
  
To add another front-end to a site, or to refresh a copied front-end:
+
This will install the built front end into 'src/<namespace>/frontend/<frontend>'.
  
  site:frontend <site> <frontend>
+
You can also choose to build just one type of resource to save time, e.g. if you only changed a template and don't want to wait for the js to be rebuilt as well:
  
== File Layout ==
+
  ./build frontend:styles <frontend>
 +
  ./build frontend:scripts <frontend>
 +
  ./build frontend:templates <frontend>
 +
  ./build frontend:assets <frontend>
 +
  ./build frontend:base <frontend>
  
Build:
+
The /build folder is organised by resource source as follows:
  
/
 
 
  |- build/
 
  |- build/
   |- .bowerrc
+
  |- core/
   |- bower.json
+
  |- frontends/
   |- console
+
    |- <frontend1>/
 +
    |- <frontend2>/
 +
    |- <frontend3>/
 +
  |- node_modules/
 +
   |- .jsbeautifyrc
 +
   |- .jshintrc
 +
   |- build
 
   |- gulpfile.js
 
   |- gulpfile.js
 
   |- packages.json
 
   |- packages.json
  |- assets/
 
    |- <name>/
 
      |- bin/
 
      |- config/
 
      |- css/
 
      |- fonts/
 
      |- img/
 
      |- js/
 
      |- less/
 
      |- scss/
 
      |- twig/
 
      |- xliff/
 
  |- node_packages/
 
  |- vendor/
 
  
Source:
+
* The /node_modules folder holds the vendor resources managed by NPM as defined in the packages.json file
 +
* The/ core folder holds the core resources provided by ark2
 +
* The /frontends folder holds the custom resources for the various integrated frontends
 +
* The /gulpfile.js defines the Gulp build scripts
 +
* The /build console runs the build scripts
 +
* The /.jsbeautifyrc and /.jshintrc files define the JavaScript coding standards used in ARK2
 +
 
 +
Within the /core folder and each custom frontend folder, the resources provided are organised in folders by resource source type:
 +
 
 +
  |- <frontend>/
 +
    |- bin/
 +
    |- config/
 +
    |- fonts/
 +
    |- images/
 +
    |- js/
 +
    |- public/
 +
    |- scss/
 +
    |- twig/
 +
    |- xliff/
 +
 
 +
== Manifest ==
 +
 
 +
The combination of resources that are built into a frontend are defined by the build manifest.json file in the root of each frontend folder, e.g. /build/frontends/ark2/manifest.json.
 +
 
 +
{
 +
    "frontend": "ark2",
 +
    "namespace": "ARK",
 +
    "options": {},
 +
    "assets": {
 +
        "fonts": {},
 +
        "images": {},
 +
        "scripts": {},
 +
        "styles": {}
 +
    },
 +
    "bin": {},
 +
    "config": {},
 +
    "templates": {},
 +
    "translations": {},
 +
    "public": {}
 +
}
 +
 
 +
Here you see the root level of the manifest defines the same resource classes as is used to organise the frontend source folder, i.e. these sections define the resources that will be packaged together to be installed into the source folder.
 +
 
 +
There are also three extra root level keys:
 +
* frontend - the name of the frontend used in the source path
 +
* namespace - the namespace of the frontend used in the source path
 +
* options - various options to be used in the build tooling, keyed by the Gulp module that uses them
 +
 
 +
The default build options are as follows:
 +
 
 +
"options": {
 +
    "uglify": {
 +
        "compress": false
 +
    },
 +
    "sass": {
 +
        "outputStyle": "compressed",
 +
        "precision": 8,
 +
        "includePaths": [
 +
            "node_modules/bootstrap-sass/assets/stylesheets"
 +
        ]
 +
    },
 +
    "sourcemaps": {
 +
        "loadMaps": true
 +
    },
 +
    "autoprefixer": {
 +
        "browsers": [
 +
            "Android 2.3",
 +
            "Android >= 4",
 +
            "Chrome >= 20",
 +
            "Firefox >= 24",
 +
            "Explorer >= 8",
 +
            "iOS >= 6",
 +
            "Opera >= 12",
 +
            "Safari >= 6"
 +
        ]
 +
    }
 +
},
 +
 
 +
Not that the autoprefixer settings are those required by Bootstrap and should not be altered.
 +
 
 +
Within each resource class, you must define what resources are to be included from the three resource sources: vendor, core and custom.
 +
 
 +
For most resource classes this defines what files to copy into the source folder, either by individual file names or (more usually for core and custom) all files in a folder:
 +
 
 +
"templates": {
 +
    "vendor": [
 +
        "symfony-collection/jquery.collection.html.twig"
 +
    ],
 +
    "core": [
 +
        "twig/**/*",
 +
        "twig/**/.*"
 +
    ],
 +
    "custom": [
 +
        "twig/**/*",
 +
        "twig/**/.*"
 +
    ]
 +
},
 +
 
 +
When copying files in this way, the order used is vendor, core, then custom, so files in core will override files in vendor and files in custom will override files in core. The vendor file paths are relative to /build/node_modules, the core files are paths relative to /build/core, and the custom files are paths relative to /build/frontends/<frontend>.
 +
 
 +
The scripts and styles groups work differently however as they must call the SASS and JavaScript build tools to compile the destination source code.
 +
 
 +
The scripts class defines a set of build targets, that is named minified javascript files to be compiled, and the javascript files that are merged to create that file. For example, to build script files ark-default.min.js and ark-map.min.js you could define the following:
 +
 
 +
"scripts": {
 +
    "ark-default": {
 +
        "vendor": [
 +
            "jquery/dist/jquery.js",
 +
            "bootstrap-sass/assets/javascripts/bootstrap.js"
 +
        ],
 +
        "core": [
 +
            "js/core-ready.js"
 +
        ],
 +
        "custom": [
 +
            "js/frontend-ready.js",
 +
        ]
 +
    },
 +
    "ark-map": {
 +
        "vendor": [
 +
            "proj4/dist/proj4.js",
 +
            "openlayers/dist/ol.js"
 +
        ],
 +
        "core": [],
 +
        "custom": [
 +
            "js/map-ready.js"
 +
        ]
 +
    }
 +
},
 +
 
 +
The javascript files are merged in the order defined in the groups, so it is recommended to explicitly list the files so that the correct order of dependencies can be defined. Sourcemap files will be created if specified in the options.
 +
 
 +
The scripts class defines a set of build targets, that is named minified CSS stylesheet files to be compiled, and the list of Gulp build streams required to built by the SASS compiler before being combined into that target file. For example, to build style files ark-default.min.css and ark-map.min.css you could define the following:
 +
 
 +
"styles": {
 +
    "ark-default": [{
 +
            "stream": "bootstrap",
 +
            "vendor": [],
 +
            "core": [],
 +
            "custom": [
 +
                "scss/vendor.scss"
 +
            ]
 +
        },
 +
        {
 +
            "stream": "main",
 +
            "vendor": [
 +
                "font-awesome/css/font-awesome.css"
 +
            ],
 +
            "core": [],
 +
            "custom": [
 +
                "scss/default.scss"
 +
            ]
 +
        }
 +
    ],
 +
    "ark-map": [{
 +
        "stream": "main",
 +
        "vendor": [
 +
            "openlayers/dist/ol.css"
 +
        ],
 +
        "core": [],
 +
        "custom": [
 +
            "scss/map.scss"
 +
        ]
 +
    }]
 +
}
 +
 
 +
The defining of streams within each target allows for the mixing of SASS and CSS source in a defined order so the cascading of stylesheets is applied in the correct order
 +
 
 +
== Templates ==
 +
 
 +
Twig template resources are organised using a convention to make dynamic mixing of templates to create many different Views possible.
 +
 
 +
The twig folder for a frontend is organised as follows:
 +
 
 +
|- twig/
 +
  |- blocks/
 +
  |- errors/
 +
  |- framework/
 +
  |- layouts/
 +
  |- pages/
 +
  |- profiler/
 +
  |- user/
 +
    |- emails/
 +
    |- layouts/
 +
 
 +
* blocks - Small chunks of twig code designed to be reused and repeated using Twig Block inheritance, e.g. form fields, widgets, etc.
 +
* layouts - Arrangements of blocks into generic or task specific layouts, e.g. grids, tables, custom content views, etc.
 +
* pages - Definitions of the sets of standard blocks to use when building the HTML document for a particular page
 +
* framework - Definition of the standard HTML document and default page blocks
 +
* errors - Standard error pages, i.e. 404, 500, etc
 +
* user - Required layouts for the user security system, e.g. user verification emails, login forms, etc.
 +
* profiler - Custom templates to be used in the Symfony profiler
 +
 
 +
The blocks used fall into three general groups:
 +
* Standard page blocks - blocks that define or override standard page elements such as scripts, navbars, etc, which are saved in separate block files.
 +
* Form blocks - blocks that define standard Symfony Form elements such as Textarea, Select, Radio, etc, which are saved in a single blocks/forms.html.twig file
 +
* Custom widget blocks - blocks that define custom widgets that do not use Symfony Forms and are saved in separate block files
  
/
+
The standard page blocks are:
|- src/
+
* head - defines the document <head> element
  |- ARK/
+
* styles - stylesheets to be included at the top of the document <head> element
    |- web/
+
* body - defines the document <body> element, usually how the navbar, sidebar, content and footer blocks are arranged in the body
      |- <frontend>/
+
* navbar - defines the navbar or header at the top of the document body
        |- .htaccess
+
* sidebar - defines the sidebar or menu at the side of the document body
        |- index.html
+
* content - defines the main content of the document body
        |- templates/
+
* footer - defines the footer at the bottom of the document body
        |- translations/
+
* scripts - scripts to be included at the bottom of the document <body> element
        |- assets/
+
* link - defines the link element used in the navbar/sidebar/footer
          |- fonts/
+
* flash - defines the page flash element
          |- images/
 
          |- scripts/
 
          |- styles/
 
      |- api/
 
      |- admin/
 
      |- core/
 
  |- <project>/
 
    |- web/
 
      |- <frontend>/
 
        |- ...
 
  
Sites:
+
The standard ARK2 page view is built in Twig as follows:
 +
* framework/doc.html.twig - Skeleton of the HTML5 document, includes undefined blocks for styles, head, body and scripts
 +
* framework/page.html.twig - Extends doc.html.twig and defines the full default set of blocks to use for styles, head, body, and scripts, includes other default blocks used by those blocks such as navbar, but excludes the content block to be used
 +
* pages/<page>.html.twig - Extends framework/page.html.twig to set the content block to be used by a page, and overrides any standard blocks with custom blocks to be used, e.g. override the standard styles or navbar blocks on this page. This is the root template rendered by a page view.
  
/
+
== Scripts ==
|- sites/
 
  |- <site>/
 
    |- config/
 
    |- files/
 
    |- schema/
 
    |- src/
 
      |- <frontend>/
 
        |- php/
 
        |- templates/
 
        |- translations/
 
    |- web/
 
      |- .htaccess
 
      |- index.php
 
      |- assets/
 
        |- <frontend>/
 
          |- fonts/
 
          |- images/
 
          |- scripts/
 
          |- styles/
 
  
== Notes ==
+
The use of Javascript in ARK2 is not very well defined or organised as yet beyond the use of jQuery and various Bootstrap-based widgets.
  
Bootstrap 3 supports both [http://lesscss.org/ Less] and [http://sass-lang.com/ Sass] templates to generate the Bootstrap CSS. Customising the appearance of Bootstrap (such as colour) usually requires modifying template variables and rebuilding the CSS. Bootstrap 4 (currently in alpha) switches to only using SASS for its templates. We should therefore choose to use the SASS version of Bootstrap 3 when building our own custom version of Bootstrap. Build tools will be provided to automate the customisation process.
+
Core Javascript resources in /build/core/js:
 +
* alert.js - Utilies for working with the ARK2/Bootstrap alerts
 +
* core-ready.js - Initialise the core javascript on document ready
 +
* form.js - Various form related tools, such as the AJAX form mappper
 +
* form-ready.js - Initialise the form javascript on document ready
 +
* location.js - An OpenLayers map picker widget as configured in the ark_map table.
 +
* map-ol4.js - An OpenLayers map widget as configured in the ark_map table.
 +
* map-ready.js - Initialise the map widget on document ready
 +
* pageedit.js - Static page editing using Summernote Editor
 +
* routing.js - The Routing module to use to generate paths from Route IDs.
 +
* utilities.js - Some utility routines, such as debounce.
  
The use of Twig templates for page layout will help separate the model and view code and allow third parties to easily modify the layout without having to alter the core code. Each Twig template will document the API contract it has with the data model, i.e. what variables are available to be used in the template.
+
Common vendor libraries used via NPM:
 +
* jquery
 +
* jquery-form
 +
* moment
 +
* bazinga-translator
 +
* select2
 +
* bootstrap
 +
* bootbox
 +
* bootstrap-datetime-picker
 +
* bootstrap-fileinput
 +
* bootstrap-table
  
The use of the Silex/Symfony Forms module will be considered. This provides dynamic form generation and validation with a Bootstrap theme.
+
== Styles ==
  
There will be separation between the ARK Admin frontend and the ARK Web frontend. The required ARK Admin frontend will be static and consistent across all ARKs, but can be modified for site specific requirements if needed (i.e. adding extra user data fields). The optional ARK Web frontend will be the dynamic generated data-driven side, configurable for every ARK. This separation will allow for ARK to run as a pure database/API backend server with basic admin and auth frontend provided without the user having to configure or enable any of the web frontend.
+
Styles use a combination of SASS and CSS as required with Bootstrap as the chosen ui style.
  
The ARK Admin frontend will provide the core UI elements for the site, i.e. the Nav Bar and Nav Menu. An initial template will be inspired by [http://startbootstrap.com/template-overviews/sb-admin-2/ SB Admin 2] (Test [http://blackrockdigital.github.io/startbootstrap-sb-admin-2/pages/index.html here]) and [https://almsaeedstudio.com/ AdminLTE] (Test [https://almsaeedstudio.com/preview here]), but greatly simplified and converted to Twig templates.
+
== Useful References ==
  
 +
* https://symfony.com/doc/3.4/form/form_customization.html
 +
* https://symfony.com/doc/3.4/reference/forms/twig_reference.html
 
* http://www.helloerik.com/the-subtle-magic-behind-why-the-bootstrap-3-grid-works
 
* http://www.helloerik.com/the-subtle-magic-behind-why-the-bootstrap-3-grid-works

Latest revision as of 14:34, 16 April 2018

Overview

ARK2 has imposed a clear separation between the backend and frontend, allowing the frontend to operate completely independently. ARK2 provides a number of standard frontend implementations, but sites can adapt or implement their own. This also supports multi-tenant mode where a single ARK2 install can be configured to run multiple ARK2 sites using different frontends.

Two options exist for building frontends:

  • Integrated frontends using the ARK2 PHP API via TWIG templates and Views defined in the config database
  • Standalone frontends using calls to the ARK2 REST API

The integrated frontends that ship with ARK2 are implemented in Bootstrap, jQuery, and Twig, the most popular and well-supported frontend libraries. Frontends can be cloned and modified for a specific site, allowing for easier customisation of ARK's appearance by third party designers. Standalone frontends could be implemented in any other technologies such as React and Vue.

The integrated frontends shipped with ARK2 are:

  • admin - A minimal Web Admin front-end for configuring an API-only server
  • ark2 - The default front-end supporting the full ARK2 feature set
  • bootstrap3 - A minimal Bootstrap 3 frontend designed as a base for custom frontends
  • bootstrap4 - A minimal Bootstrap 4 frontend designed as a base for custom frontends
  • flat - A simple, non-js, read-only front-end designed for scraping, e.g. for static archiving or web-bots

The integrated frontends require a build process to assemble the required frontend resources into a distributable form. This page documents the process required to do so.

The resources built into a frontend can be obtained from three sources:

  • Vendor resources: Third-party libraries from outside ARK2, such as Bootstrap, jQuery, and FontAwesome
  • Core resources: Standard resources provided by ARK2 for interacting with ARK2 features, e.g. common widgets, form layouts, etc
  • Custom resources: Custom resources developed for a particular frontend

The build process uses tooling to combine these resources into a single source package that is able to be distributed.

Source

Packaged frontend resources are saved in the src directory by Namespace and Name:

 src/<namespace>/frontend/<frontend>

For example, the main ark2 frontend in the ARK name space is stored in:

 src/ARK/frontend/ark2

Within each frontend source folder, the source is organised by general type:

 |- <frontend>/
   |- assets/
     |- fonts/
     |- images/
     |- scripts/
     |- styles/
   |- bin/
     |- console
   |- config/
     |- credentials.json
     |- database.json
     |- site.json
   |- public/
     |- .htaccess
     |- index.php
   |- templates/
   |- translations/

The following source types generally use the default source and remain unchanged:

  • bin - Any executables used by the frontend, usually just the site console
  • config - The default config files for the frontend, the main site.json file, the database.json file, and the credentials.json file
  • public - The default contents of the public webroot folder, including the index.php file

The following source types are usually modified for a custom frontend and rebuilt using the build tooling when changed:

  • assets - The public assets for the frontend that get included under the public webroot, i.e. fonts, images, scripts and stylesheets
  • templates - The twig templates used to render the html
  • translations - Vendor provided translation files

Customising the scripts and stylesheets requires various tooling which is covered in the next section.

Build Tooling

Build tooling for frontend resources is required for a number of reasons:

  • Bootstrap can be customised most easily by changing variables used in the Sass templates, which then requires a build step to compile them into CSS
  • Production deployment is more efficient if CSS and JS is stripped, merged and minified, while development is easier if a map is generated for the original code
  • Resources from multiple sources need to be combined into a single package
  • Multiple versions and combinations of resources may be required for performance reasons
  • NPM library management downloads the entire package, not just the resources required, an extra step is required to copy just the required resources into the source folder
  • All the steps required for packaging and release management can be automated, e.g. clean, compile, tag, package, etc
  • The build tooling for the default ARK bootstrap and twig theme can be generalised to allow clients to build and deploy their own customised themes with minimal effort

The build tooling works as follows:

  • All build tooling and resources are isolated in the /build folder so it can be excluded from any release packages or production deployments
  • Nothing in the /build folder may be depended on by any code outside the /build folder or required for running ARK itself
  • Node, npm, Gulp and Symfony Console are used to run the tooling
  • NPM is used to manage the Node packages used which are installed into /build/node_packages
  • Node packages are used both for the build tools themselves and for vendor libraries providing resources
  • Gulp is used to ensure the build scripts run cross-platform
  • Symfony Console is used to manage and run the build tasks
  • Running tasks will only work inside the /build folder, trying to run outside the build folder will fail
  • The resources required to be built for a frontend are defined in a manifest file
  • The built frontend resources are then installed into the frontend source directory
  • Later, each configured site in the ARK2 install links to the frontend it wishes to use

Build

The front end build is managed by the Build Console. The Build console is in the 'build/' folder and is used to build/rebuild/install a front-end if required. To run the console you must be in the build folder:

 cd build
 ./build

Running the console without a chosen command will show the list of available commands:

ARK Build Console 1.9.80

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands:
  help                Displays help for a command
  list                Lists commands
 cache
  cache:clear         Clear the system cache
 database
  database:reverse    Reverse engineer an existing database as DoctrineXML
 frontend
  frontend:all        Build an ARK frontend. (Args: frontend)
  frontend:assets     Build the frontend assets (Fonts, Images, Scripts, Styles). (Args: frontend)
  frontend:base       Build the frontend base (Bin, Config, Templates, Translations, Web). (Args: frontend)
  frontend:create     Create a new ARK frontend (Args: namespace, frontend).
  frontend:scripts    Build the frontend scripts (JavaScript). (Args: frontend)
  frontend:styles     Build the frontend styles (CSS and SASS). (Args: frontend)
  frontend:templates  Build the frontend templates (Twig). (Args: frontend)

The first time you need to build/rebuild a frontend, you need to install the required Node packages. You will need to have installed Node and NPM on your system first.

 npm install

To update an existing installation, especially after an ARK upgrade:

 npm update

To check the status of your Node environment (including available updates), run:

 npm doctor

To install a new node package to use in the frontend, run:

 npm install <package>

To build a frontend run the frontend command with the name of the frontend:

 ./build frontend:all <frontend>

This will install the built front end into 'src/<namespace>/frontend/<frontend>'.

You can also choose to build just one type of resource to save time, e.g. if you only changed a template and don't want to wait for the js to be rebuilt as well:

 ./build frontend:styles <frontend>
 ./build frontend:scripts <frontend>
 ./build frontend:templates <frontend>
 ./build frontend:assets <frontend>
 ./build frontend:base <frontend>

The /build folder is organised by resource source as follows:

|- build/
  |- core/
  |- frontends/
    |- <frontend1>/
    |- <frontend2>/
    |- <frontend3>/
  |- node_modules/
  |- .jsbeautifyrc
  |- .jshintrc
  |- build
  |- gulpfile.js
  |- packages.json
  • The /node_modules folder holds the vendor resources managed by NPM as defined in the packages.json file
  • The/ core folder holds the core resources provided by ark2
  • The /frontends folder holds the custom resources for the various integrated frontends
  • The /gulpfile.js defines the Gulp build scripts
  • The /build console runs the build scripts
  • The /.jsbeautifyrc and /.jshintrc files define the JavaScript coding standards used in ARK2

Within the /core folder and each custom frontend folder, the resources provided are organised in folders by resource source type:

 |- <frontend>/
   |- bin/
   |- config/
   |- fonts/
   |- images/
   |- js/
   |- public/
   |- scss/
   |- twig/
   |- xliff/

Manifest

The combination of resources that are built into a frontend are defined by the build manifest.json file in the root of each frontend folder, e.g. /build/frontends/ark2/manifest.json.

{
    "frontend": "ark2",
    "namespace": "ARK",
    "options": {},
    "assets": {
        "fonts": {},
        "images": {},
        "scripts": {},
        "styles": {}
    },
    "bin": {},
    "config": {},
    "templates": {},
    "translations": {},
    "public": {}
}

Here you see the root level of the manifest defines the same resource classes as is used to organise the frontend source folder, i.e. these sections define the resources that will be packaged together to be installed into the source folder.

There are also three extra root level keys:

  • frontend - the name of the frontend used in the source path
  • namespace - the namespace of the frontend used in the source path
  • options - various options to be used in the build tooling, keyed by the Gulp module that uses them

The default build options are as follows:

"options": {
    "uglify": {
        "compress": false
    },
    "sass": {
        "outputStyle": "compressed",
        "precision": 8,
        "includePaths": [
            "node_modules/bootstrap-sass/assets/stylesheets"
        ]
    },
    "sourcemaps": {
        "loadMaps": true
    },
    "autoprefixer": {
        "browsers": [
            "Android 2.3",
            "Android >= 4",
            "Chrome >= 20",
            "Firefox >= 24",
            "Explorer >= 8",
            "iOS >= 6",
            "Opera >= 12",
            "Safari >= 6"
        ]
    }
},

Not that the autoprefixer settings are those required by Bootstrap and should not be altered.

Within each resource class, you must define what resources are to be included from the three resource sources: vendor, core and custom.

For most resource classes this defines what files to copy into the source folder, either by individual file names or (more usually for core and custom) all files in a folder:

"templates": {
    "vendor": [
        "symfony-collection/jquery.collection.html.twig"
    ],
    "core": [
        "twig/**/*",
        "twig/**/.*"
    ],
    "custom": [
        "twig/**/*",
        "twig/**/.*"
    ]
},

When copying files in this way, the order used is vendor, core, then custom, so files in core will override files in vendor and files in custom will override files in core. The vendor file paths are relative to /build/node_modules, the core files are paths relative to /build/core, and the custom files are paths relative to /build/frontends/<frontend>.

The scripts and styles groups work differently however as they must call the SASS and JavaScript build tools to compile the destination source code.

The scripts class defines a set of build targets, that is named minified javascript files to be compiled, and the javascript files that are merged to create that file. For example, to build script files ark-default.min.js and ark-map.min.js you could define the following:

"scripts": {
    "ark-default": {
        "vendor": [
            "jquery/dist/jquery.js",
            "bootstrap-sass/assets/javascripts/bootstrap.js"
        ],
        "core": [
            "js/core-ready.js"
        ],
        "custom": [
            "js/frontend-ready.js",
        ]
    },
    "ark-map": {
        "vendor": [
            "proj4/dist/proj4.js",
            "openlayers/dist/ol.js"
        ],
        "core": [],
        "custom": [
            "js/map-ready.js"
        ]
    }
},

The javascript files are merged in the order defined in the groups, so it is recommended to explicitly list the files so that the correct order of dependencies can be defined. Sourcemap files will be created if specified in the options.

The scripts class defines a set of build targets, that is named minified CSS stylesheet files to be compiled, and the list of Gulp build streams required to built by the SASS compiler before being combined into that target file. For example, to build style files ark-default.min.css and ark-map.min.css you could define the following:

"styles": {
    "ark-default": [{
            "stream": "bootstrap",
            "vendor": [],
            "core": [],
            "custom": [
                "scss/vendor.scss"
            ]
        },
        {
            "stream": "main",
            "vendor": [
                "font-awesome/css/font-awesome.css"
            ],
            "core": [],
            "custom": [
                "scss/default.scss"
            ]
        }
    ],
    "ark-map": [{
        "stream": "main",
        "vendor": [
            "openlayers/dist/ol.css"
        ],
        "core": [],
        "custom": [
            "scss/map.scss"
        ]
    }]
}

The defining of streams within each target allows for the mixing of SASS and CSS source in a defined order so the cascading of stylesheets is applied in the correct order

Templates

Twig template resources are organised using a convention to make dynamic mixing of templates to create many different Views possible.

The twig folder for a frontend is organised as follows:

|- twig/
  |- blocks/
  |- errors/
  |- framework/
  |- layouts/
  |- pages/
  |- profiler/
  |- user/
    |- emails/
    |- layouts/
  • blocks - Small chunks of twig code designed to be reused and repeated using Twig Block inheritance, e.g. form fields, widgets, etc.
  • layouts - Arrangements of blocks into generic or task specific layouts, e.g. grids, tables, custom content views, etc.
  • pages - Definitions of the sets of standard blocks to use when building the HTML document for a particular page
  • framework - Definition of the standard HTML document and default page blocks
  • errors - Standard error pages, i.e. 404, 500, etc
  • user - Required layouts for the user security system, e.g. user verification emails, login forms, etc.
  • profiler - Custom templates to be used in the Symfony profiler

The blocks used fall into three general groups:

  • Standard page blocks - blocks that define or override standard page elements such as scripts, navbars, etc, which are saved in separate block files.
  • Form blocks - blocks that define standard Symfony Form elements such as Textarea, Select, Radio, etc, which are saved in a single blocks/forms.html.twig file
  • Custom widget blocks - blocks that define custom widgets that do not use Symfony Forms and are saved in separate block files

The standard page blocks are:

  • head - defines the document <head> element
  • styles - stylesheets to be included at the top of the document <head> element
  • body - defines the document <body> element, usually how the navbar, sidebar, content and footer blocks are arranged in the body
  • navbar - defines the navbar or header at the top of the document body
  • sidebar - defines the sidebar or menu at the side of the document body
  • content - defines the main content of the document body
  • footer - defines the footer at the bottom of the document body
  • scripts - scripts to be included at the bottom of the document <body> element
  • link - defines the link element used in the navbar/sidebar/footer
  • flash - defines the page flash element

The standard ARK2 page view is built in Twig as follows:

  • framework/doc.html.twig - Skeleton of the HTML5 document, includes undefined blocks for styles, head, body and scripts
  • framework/page.html.twig - Extends doc.html.twig and defines the full default set of blocks to use for styles, head, body, and scripts, includes other default blocks used by those blocks such as navbar, but excludes the content block to be used
  • pages/<page>.html.twig - Extends framework/page.html.twig to set the content block to be used by a page, and overrides any standard blocks with custom blocks to be used, e.g. override the standard styles or navbar blocks on this page. This is the root template rendered by a page view.

Scripts

The use of Javascript in ARK2 is not very well defined or organised as yet beyond the use of jQuery and various Bootstrap-based widgets.

Core Javascript resources in /build/core/js:

  • alert.js - Utilies for working with the ARK2/Bootstrap alerts
  • core-ready.js - Initialise the core javascript on document ready
  • form.js - Various form related tools, such as the AJAX form mappper
  • form-ready.js - Initialise the form javascript on document ready
  • location.js - An OpenLayers map picker widget as configured in the ark_map table.
  • map-ol4.js - An OpenLayers map widget as configured in the ark_map table.
  • map-ready.js - Initialise the map widget on document ready
  • pageedit.js - Static page editing using Summernote Editor
  • routing.js - The Routing module to use to generate paths from Route IDs.
  • utilities.js - Some utility routines, such as debounce.

Common vendor libraries used via NPM:

  • jquery
  • jquery-form
  • moment
  • bazinga-translator
  • select2
  • bootstrap
  • bootbox
  • bootstrap-datetime-picker
  • bootstrap-fileinput
  • bootstrap-table

Styles

Styles use a combination of SASS and CSS as required with Bootstrap as the chosen ui style.

Useful References