|
|
| (23 intermediate revisions by the same user not shown) |
| Line 6: |
Line 6: |
| | * Full code re-write to modern standards using modern components | | * Full code re-write to modern standards using modern components |
| | * Separate the ARK Database backend from the ARK Web frontend | | * Separate the ARK Database backend from the ARK Web frontend |
| − | * Implement a modern RESTful API to allow other frontends and apps to access and update the ARK Database | + | * Implement a modern REST API to allow other frontends and apps to access and update the ARK Database |
| | * Simplify the setup and configuration of ARK by moving the config into the database and providing online config tools | | * Simplify the setup and configuration of ARK by moving the config into the database and providing online config tools |
| | * Improve the overall performance and data integrity of ARK | | * Improve the overall performance and data integrity of ARK |
| Line 13: |
Line 13: |
| | Modern frontend | | Modern frontend |
| | * HTML5 | | * HTML5 |
| − | * Bootstrap 3 based | + | * Bootstrap based |
| | * Twig templates | | * Twig templates |
| − | * Front Controller / Single Page model | + | * Front Controller model |
| | * Config driven pages views | | * Config driven pages views |
| | | | |
| | Modern backend | | Modern backend |
| − | * Modern RESTful API to access and update all ARK data | + | * Full REST API to access and update all ARK data |
| − | * Database abstraction | + | * Database abstraction and Object Relational Mapping |
| − | * Config driven model | + | * Config driven data schema |
| | + | * Controlled Vocabularies and Linked Open Data |
| | * User Authentication via internal user/password and external OAuth2 providers (Facebook, Google, etc) | | * User Authentication via internal user/password and external OAuth2 providers (Facebook, Google, etc) |
| | * User Authorisation via Role Based Access Control (RBAC) using hierarchical Roles and Permissions structure | | * User Authorisation via Role Based Access Control (RBAC) using hierarchical Roles and Permissions structure |
| | + | * Field-level data access control |
| | + | * Data Workflows in conjunction with User Authorisation control |
| | | | |
| | == Documentation == | | == Documentation == |
| Line 30: |
Line 33: |
| | | | |
| | * [[ARK2/Design|Design]] - High Level Design Decisions | | * [[ARK2/Design|Design]] - High Level Design Decisions |
| | + | * [[ARK2/Technical|Technical]] - Technical details, Development tools and procedures |
| | + | * [[ARK2/The_ARK_Way|The ARK Way]] - Web Development - The ARK2 Way |
| | + | * [[ARK2/Install|Install]] - Installation Instructions |
| | * [[ARK2/Architecture|Architecture]] - System Architecture | | * [[ARK2/Architecture|Architecture]] - System Architecture |
| − | * [[ARK2/Development|Development]] - Development tools and procedures | + | * [[ARK2/Database|Database]] - Database / ORM details |
| | + | * [[ARK2/Cache|Cache]] - Cache details |
| | * [[ARK2/Model|Model]] - Data model / Schema | | * [[ARK2/Model|Model]] - Data model / Schema |
| | + | * [[ARK2/View|View]] - Views on the Data Model |
| | + | * [[ARK2/Spatial|Spatial]] - Spatial data |
| | + | * [[ARK2/Vocabulary|Vocabulary]] - Controlled Vocabularies |
| | + | * [[ARK2/Files|File Management]] - File, Image, and other Media Management |
| | + | * [[ARK2/Localization|Localization]] - Internationalization, Localization, and Translation |
| | + | * [[ARK2/Security|Security]] - Security, Authentication, Authorisation, User Management |
| | * [[ARK2/API|API]] - REST API implementation | | * [[ARK2/API|API]] - REST API implementation |
| | * [[ARK2/Frontend|Frontend]] - Web Frontend | | * [[ARK2/Frontend|Frontend]] - Web Frontend |
| − | * [[ARK2/Database|Database]] - Database details | + | * [[ARK2/Templates|Templates]] - Using Twig for the Web Frontend |
| − | * [[ARK2/Localization|Localization]] - Internationalization, Localization, and Translation | + | * [[ARK2/Console|Console]] - Admin Consoles |
| − | * [[ARK2/Security|Security]] - Security, Authentication, Authorisation, User Management | + | * [[ARK2/Admin|Admin]] - Admin Frontend |
| − | | + | * [[ARK2/Branding|Branding]] - Branding of the ARK Project, Products, and Service |
| − | == Design ==
| |
| − | | |
| − | High level design decisions for ARK2.
| |
| − | | |
| − | === Supported Platforms ===
| |
| − | | |
| − | ARK will only actively support platforms that are actively supported by their maintainers. ARK may work on earlier versions but this is not guaranteed.
| |
| − | | |
| − | * HTML5 will be used | |
| − | * Browser support restricted to those supported by Bootstrap 3
| |
| − | * PHP: A minimum of v5.6 will be supported (5.6 is in Security Support, 5.7 in active support, see http://php.net/supported-versions.php), v7 will be supported.
| |
| − | * MySQL/MariaDB v5.5 or later (lowest supported MySQL)
| |
| − | * PostgreSQL v9.2 or later
| |
| − | * SQLite 3.7.11 or later (required for multiple inserts)
| |
| − | * mod_rewrite will be required
| |
| − | * PHP intl extension and ICU will be required
| |
| − | | |
| − | For more details see the [[ARK2/Development]] page.
| |
| − | | |
| − | === Framework ===
| |
| − | | |
| − | It is proposed to implement a new RESTful Request/Route/Response skeleton using a Front Controller model and token-based security, based on an external micro-framework and components adhering to the PSR standards and managed via Composer. This will reduce the amount of code maintained internally, update the code-base to modern web-app design principals, and provide a degree of future-proofing by allowing switching of components.
| |
| − | | |
| − | The [http://silex.sensiolabs.org/ Silex] micro-framework and [http://symfony.com/ Symphony] components have been selected due to their high quality, long support history, and wide use and support
| |
| − | | |
| − | For more details see the [[ARK2/Development]] page.
| |
| − | | |
| − | === Database Abstraction ===
| |
| − | | |
| − | ARK2 will support the use of MySQL, PostgreSQL or SQLite as the database.
| |
| − | | |
| − | In ARK1, PDO is used to directly access only MySQL databases, and DB access statements are widely spread through the code base and manually coded. While PDO abstracts the connection, it doesn't abstract the SQL dialect so adding support for other databases such as Postgres or SQLite would require considerable work. It also makes migration to proper transaction support and performance improvements difficult, and is a security risk due to programmer error.
| |
| − | | |
| − | A Database Abstraction Layer (DAL) can abstract away the differences in SQL between database systems, and also provide Query Builders, Schema Management, and Migration tools to address the other issues. Most are built on PDO and can seamlessly integrate with legacy code to make for an easier migration path. The [http://doctrine-orm.readthedocs.io/projects/doctrine-dbal/en/latest/ Doctrine DBAL] DAL has been selected due to it's use in Symfony and the ORM extension being a Data Mapping ORM.
| |
| − | | |
| − | More details can be found on the [[ARK2/Database]] page.
| |
| − | | |
| − | === Security ===
| |
| − | | |
| − | ARK currently uses PEAR LiveUser for user authentication, authorisation, and user management, but this hasn't been updated since 2010. It is a security risk, and also lacks many features like federated login. The ARK API currently uses plain text user and password in the request URL which is insecure. ARK2 will require a new security solution, especially for the API calls from client apps. This solution will be token based, support username/password and OAuth2 authentication, and use Role-Based Access Control for authorization. HTTPS will be required for the API.
| |
| − | | |
| − | For more details, see the [[ARK2/Security|Security]] page.
| |
| − | | |
| − | === REST / HATEOAS / Hypermedia ===
| |
| − | | |
| − | An evolution of the ARK data model and API to try realise the [http://ark.lparchaeology.com/hypertext/ full ARK vision] will be based arround the Hypermedia concepts of [https://en.wikipedia.org/wiki/Representational_state_transfer REST] and [https://en.wikipedia.org/wiki/HATEOAS HATEOAS] as developed by Roy Fielding in his doctoral thesis in 2000. These concepts include resources, relationships, state, and discoverability, and are closely related to the Semantic Web. ARK modules will evolve to represent Resources that can be linked in together in the current flat relationship structure, or organised into configurable hierarchies such as the default Site/Module/Item mostly used by ARK instances. These concepts will be most easily exposed through a RESTful API.
| |
| − | | |
| − | For more details see the [[ARK2/API|API]] page.
| |
| − | | |
| − | === Frontend ===
| |
| − | | |
| − | The frontend will be migrated to [http://getbootstrap.com/ Bootstrap], [https://jquery.com/ jQuery], and [http://twig.sensiolabs.org/ Twig], the most popular and well-supported frontend ui component and template systems. This will allow for easier customisation of ARK's appearance by third parties.
| |
| − | | |
| − | For more details, see the [[ARK2/Frontend|Frontend]] page.
| |
| − | | |
| − | === Migration ===
| |
| − | | |
| − | A migration process from ARK 1 to ARK 2 will be provided.
| |
| − | | |
| − | Data migration. Existing tables will need to change from MyISAM to InnoDB. Change in place carries a degree of risk of data loss if the migration fails part way. Attempting to restart failed migrations is also prone to error. To protect users data, a new database will be created with new tables and the data copied across. Should migration fail users will easily be able to roll back to their old install, or keep retrying the migration until it does succeed. In effect the ARK init script will be run, followed by the migration script.
| |
| − | | |
| − | User migration. Users will be migrated from LiveUser to the new RBAC system. This will require a compatible default user config.
| |
| − | | |
| − | Config migration. A config migration script will be provided, but may require adapting for individual ARKs.
| |
| − | | |
| − | === Sysadmin Console ===
| |
| − | | |
| − | An sysadmin console will be provided for use on the command line. This will provide a number of tools:
| |
| − | * Database administration, such as creation, migration, backup, etc
| |
| − | * MultiArk installs
| |
| − | * ARK wide alerts
| |
| − | * Maintenance mode (immediate and schedule)
| |
| − | * Upgrade tools
| |
| − | * etc
| |
| − | | |
| − | Equivalents for some of these functions will be provided in an ARK Sysadmin panel (separate to the ARK Admin panel):
| |
| − | * ARK wide alerts
| |
| − | * Maintenance mode (immediate and schedule)
| |
| − | * etc
| |
| − | | |
| − | === Translations / Localisation ===
| |
| − | | |
| − | A key to providing Ark-As-A-Service will be translating the user interface and schemas into as many languages as possible to maximise the potential user base. ARK1 does not have the tools to make this process easy to perform or manage, and it would be a waste of resources to build them. It is recommended to use one of the existing online open-source translation projects to crowd-source the translations. This will allow interested parties and potential clients to translate ARK for themselves and to grow a local community to support ARK in their country.
| |
| − | | |
| − | Changes will be made to the translation process to bring ARK into line with industry best practices and tooling, allowing for common features such as correct plural forms. This will be based on the Symfony Translation componant.
| |
| − | | |
| − | More details can be found on the [[ARK2/Localization]] page.
| |
| − | | |
| − | === Site Module ===
| |
| − | | |
| − | A new core module for Sites will be added to support recording of metadata about a site. The data schema will be configurable the same as other modules, but certain default fields will be enforced. This will be a special case module that does not appear in the modules for a site.
| |
| − | | |
| − | === Address Book / Actors / Users ===
| |
| − | | |
| − | Currently actors are allocated at a site level. This may not scale very well to a major corporate install, for example LP or WCC, with a staff of 30-50 and hundreds of sites. Repeatedly allocating new actor IDs for each site could result in a user possibly having hundreds of actor IDs which could make reporting harder, site creation more effort, and is not very RESTful.
| |
| − | | |
| − | It is proposed instead to to rename the Address Book module as the Actors module and add a new global level of actors in addition to site-specific actors. Users will be in the global list, and allocated to sites with roles. Choice can be made between global level roles across all sites, or site specific roles. Non-user global actors can also be defined, can be referred to without allocation. Local site actors can be created.
| |
| − | | |
| − | It is also proposed to have the Actors module be the only source of actors for the Action dataclass, simplifying the storage and access.
| |
| − | | |
| − | === Action Logging / Activity Streams / Gamification ===
| |
| − | | |
| − | All user actions and events in ARK will be logged, enabling an standard Activity Stream at user level that can support gamification. The actual gamification is considered outside of the scope of core ARK2. Some inspiration for the implementation can be taken from Event Sourcing, but a full implementation will not be attempted.
| |
| − | | |
| − | * http://martinfowler.com/eaaDev/EventSourcing.html
| |
| − | * http://activitystrea.ms/
| |
| − | * https://www.w3.org/TR/activitystreams-core/
| |
| − | * https://github.com/barnabywalters/php-activitystreams
| |
| − | * https://github.com/redpanda/ActivityStreams
| |
| − | * http://jwage.com/post/54480997504/building-activity-streams
| |
| − | * https://dev.playlyfe.com/guides/getting-started.html
| |
| − | | |
| − | === Command Bus Architecture / Application Services ===
| |
| − | | |
| − | A Command Bus / Event Bus architecture will be implemented to support execution or queuing of synchronous and asynchronous Commands and Events. Console Commands will be wrappers around Commands that parse input from the command line.
| |
| − | | |
| − | * http://php-and-symfony.matthiasnoback.nl/2015/01/a-wave-of-command-buses/
| |
| − | * https://github.com/SimpleBus/
| |
| − | * http://tactician.thephpleague.com/
| |
| − | * http://culttt.com/2014/11/10/creating-using-command-bus/
| |
| − | | |
| − | === Workflow Management ===
| |
| − | | |
| − | Stretch goal. Needed for Avalon. Packages exist to define workflows using state machines.
| |
| − | | |
| − | Possible workflow scenarios:
| |
| − | * Post-ex
| |
| − | * Sample taking
| |
| − | * Avalon jobs and documents
| |
| − | * Data checking
| |
| − | | |
| − | === Document Management ===
| |
| − | | |
| − | Stretch goal. Needed for Avalon. Management of documents and versions a la Sharepoint. Try use CMIS standard as used in LibreOffice.
| |
| − | | |
| − | * https://www.alfresco.com/cmis
| |
| − | * https://packagist.org/packages/dkd/php-cmis
| |
| − | | |
| − | === CMS / Blog / Project Websites ===
| |
| − | | |
| − | Either provide an OoB Wordpress integration and host client websites, or add features to ARK to provide the basic project website features (home page, blog, contact us, calendar).
| |
| − | | |
| − | * https://github.com/bolt/bolt CMS built using Silex/Symfony
| |
| − | | |
| − | === Errors ===
| |
| − | | |
| − | Standardised error codes will be used, based on the JSON API standard format. Error codes will be stored in a database table and exported via the API, with actual error messages translated via the standard methods, with detailed debug/help available on the ARK wiki via standardised links. Fatal errors will be thrown as exceptions, with the Controller responsible for catching the error and reporting it in the appropriate manner, i.e. via web or api. Non-fatal errors such as validation errors can be batched before return. While numeric error codes are convenient, they make code hard to read and debugging tracebacks harder, so all error conditions should be explicitly commented in the code, and error codes should be as unique as possible..
| |
| − | | |
| − | === Matrix / Graph ===
| |
| − | | |
| − | PHP:
| |
| − | * https://github.com/clue/graph
| |
| − | * https://github.com/graphp/algorithms
| |
| − | * https://github.com/cpettitt/graphlib/wiki/API-Reference
| |
| − | | |
| − | == Branding / Community ==
| |
| − | | |
| − | Two potential issues suggest an evolution of the ARK brand is required
| |
| − | * Building a development and support community may be easier if ARK is branded as a stand-alone project, rather than seen as owned/controlled by LP Archaeology
| |
| − | * Extending use of ARK and Hosted ARK to areas outside archaeology may be held back by emphasising the archaeology aspect in the branding
| |
| − | | |
| − | Branding as something like 'The ARK Project, sponsored by LP Archaeology', and coining a bacronym like 'ARK Recording Kit' might solve these issues.
| |
| − | | |
| − | The Hosted ARK would need a separate identity to the development project to keep the Open Source / Commercial split clear. A simple .com vs .org difference is probably not clear enough. Examples include:
| |
| − | * Salesforce.com vs Force.com
| |
| − | * Mediawiki vs Wikia
| |
| − | * Wordpress .com vs .org
| |
| − | | |
| − | Branding would thus consist of three parts:
| |
| − | * The project
| |
| − | * The products
| |
| − | * The service
| |
| − | | |
| − | The branding would need to be distinctive but consistent to make it clear they are part of a cohesive whole.
| |
| − | | |
| − | Words with Ark / Arc in them (but not arch) for possible project or theme names:
| |
| − | * Archive / ARKhive
| |
| − | * Arctic (very white/light theme?)
| |
| − | * Arcadia
| |
| − | * [https://en.wikipedia.org/wiki/Arkose Arkose] (type of sediment)
| |
| − | * Arkaeology (available in .com, .org, .net!)
| |
| − | * Arcade
| |
| − | * Archaic / Arcane / Arcanum / Arcana (more apt for ARK v1 ;-)
| |
| − | * Arcuate / Arcuated (Arc/bow shaped)
| |
| − | * Architrave
| |
| − | * Architect
| |
| − | * Archipeligo
| |
| − | * Archosaur
| |
| − | * Arktivity...
| |
