Difference between revisions of "ARK2/Files"
From ARK
(→File Management) |
(→File Management) |
||
| (4 intermediate revisions by the same user not shown) | |||
| Line 6: | Line 6: | ||
* The file system is abstracted using [http://flysystem.thephpleague.com/ FlySystem] allowing files to be transparently saved locally or to cloud providers. | * The file system is abstracted using [http://flysystem.thephpleague.com/ FlySystem] allowing files to be transparently saved locally or to cloud providers. | ||
* File versioning is supported. | * File versioning is supported. | ||
| − | * Image manipulation is abstracted using [https://github.com/avalanche123/Imagine Imagine] which supports ImageMagick or GD. | + | * Image manipulation is abstracted using [http://glide.thephpleague.com/ Glide] and [https://github.com/avalanche123/Imagine Imagine] which supports ImageMagick or GD. |
* Thumbnails are stored separately from the master files to make backups, archiving, and regeneration easier | * Thumbnails are stored separately from the master files to make backups, archiving, and regeneration easier | ||
* Multiple thumbnail profiles can be configured | * Multiple thumbnail profiles can be configured | ||
* Thumbnails will be generated asynchronously, so can either be queued for batch processing, delayed until a suitable time window, or only generated when needed. | * Thumbnails will be generated asynchronously, so can either be queued for batch processing, delayed until a suitable time window, or only generated when needed. | ||
| − | The Media Types and File Types supported are configurable, with standard types provided by default: | + | The Media Types and File Types supported are configurable, with the [https://www.iana.org/assignments/media-types/media-types.xhtml IANA standard types] provided by default: |
* Image (jpg, png, etc) | * Image (jpg, png, etc) | ||
* Audio (mp3, etc) | * Audio (mp3, etc) | ||
* Video (mkv, etc) | * Video (mkv, etc) | ||
* Text (txt, etc) | * Text (txt, etc) | ||
| − | * | + | * Application (pdf, odt, ods, etc) |
| − | |||
| − | |||
In multi-tenant mode, the files for each ARK instance are stored separately under the 'sites' folder, e.g. 'sites/mysite/files'. | In multi-tenant mode, the files for each ARK instance are stored separately under the 'sites' folder, e.g. 'sites/mysite/files'. | ||
| Line 26: | Line 24: | ||
- files | - files | ||
|- download | |- download | ||
| + | |- <user> | ||
|- upload | |- upload | ||
| + | |- <user> | ||
|- tmp | |- tmp | ||
| + | |- <user> | ||
|- data | |- data | ||
|- <mediatype> | |- <mediatype> | ||
|- <tokens> | |- <tokens> | ||
| − | |- | + | |- cache |
|- <profile> | |- <profile> | ||
|- <mediatype> | |- <mediatype> | ||
|- <tokens> | |- <tokens> | ||
| − | where the < | + | where the <mediatype> is as defined in the configuration, and the <token> is a configurable token to split the folders into manageable sizes. By default the <token> will be split by file id in groups of 1000, so subfolders for 0, 1000, 2000, 3000, etc. |
The individual file names will be in a standard format. Flysystem abstracts folders and file names as a relative path, which will take the form: | The individual file names will be in a standard format. Flysystem abstracts folders and file names as a relative path, which will take the form: | ||
files/data/<mediatype>/<token>/<id>.<revision>.<suffix> | files/data/<mediatype>/<token>/<id>.<revision>.<suffix> | ||
| − | files/ | + | files/cache/<profile>/<mediatype>/<token>/<id>.<revision>.<suffix> |
There will be no direct file or image links, all files are considered private and requests to the mapped URL will be security checked. The mapped URL forms will be: | There will be no direct file or image links, all files are considered private and requests to the mapped URL will be security checked. The mapped URL forms will be: | ||
| Line 47: | Line 48: | ||
A custom Flysystem manager is provided that allows different filesystems to be transparently mounted at different points in the file path. For example, the root files/ path is mounted locally, but files/data/ is mounted on Amazon S3. | A custom Flysystem manager is provided that allows different filesystems to be transparently mounted at different points in the file path. For example, the root files/ path is mounted locally, but files/data/ is mounted on Amazon S3. | ||
| + | |||
| + | == Metadata == | ||
| + | |||
| + | Full metadata is supported in line with Dublin Core. | ||
| + | |||
| + | == Versions == | ||
== Design Work == | == Design Work == | ||
| − | |||
| − | |||
| − | |||
* Data downloads / exports | * Data downloads / exports | ||
| − | * Documentation | + | * Documentation files |
| − | * Temp files | + | * Temp files with expiry |
* Mapping files | * Mapping files | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
Very similar to http://documentation.concrete5.org/developers/working-with-files-and-the-file-manager/overview | Very similar to http://documentation.concrete5.org/developers/working-with-files-and-the-file-manager/overview | ||
Latest revision as of 15:51, 29 April 2017
File Management
File and Media management have seen major changes in ARK2, with a more flexible system implemented.
- Files are now a core Module instead of a dataclass and look-up table, allowing for flexible metadata based on file type, and using files as either data properties or relationships.
- The file system is abstracted using FlySystem allowing files to be transparently saved locally or to cloud providers.
- File versioning is supported.
- Image manipulation is abstracted using Glide and Imagine which supports ImageMagick or GD.
- Thumbnails are stored separately from the master files to make backups, archiving, and regeneration easier
- Multiple thumbnail profiles can be configured
- Thumbnails will be generated asynchronously, so can either be queued for batch processing, delayed until a suitable time window, or only generated when needed.
The Media Types and File Types supported are configurable, with the IANA standard types provided by default:
- Image (jpg, png, etc)
- Audio (mp3, etc)
- Video (mkv, etc)
- Text (txt, etc)
- Application (pdf, odt, ods, etc)
In multi-tenant mode, the files for each ARK instance are stored separately under the 'sites' folder, e.g. 'sites/mysite/files'.
The directory structure for each instance is as follows:
- files
|- download
|- <user>
|- upload
|- <user>
|- tmp
|- <user>
|- data
|- <mediatype>
|- <tokens>
|- cache
|- <profile>
|- <mediatype>
|- <tokens>
where the <mediatype> is as defined in the configuration, and the <token> is a configurable token to split the folders into manageable sizes. By default the <token> will be split by file id in groups of 1000, so subfolders for 0, 1000, 2000, 3000, etc.
The individual file names will be in a standard format. Flysystem abstracts folders and file names as a relative path, which will take the form:
files/data/<mediatype>/<token>/<id>.<revision>.<suffix> files/cache/<profile>/<mediatype>/<token>/<id>.<revision>.<suffix>
There will be no direct file or image links, all files are considered private and requests to the mapped URL will be security checked. The mapped URL forms will be:
files/<id>/<name> files/<id>/<basename>_<profile>.<suffix>
A custom Flysystem manager is provided that allows different filesystems to be transparently mounted at different points in the file path. For example, the root files/ path is mounted locally, but files/data/ is mounted on Amazon S3.
Metadata
Full metadata is supported in line with Dublin Core.
Versions
Design Work
- Data downloads / exports
- Documentation files
- Temp files with expiry
- Mapping files
Very similar to http://documentation.concrete5.org/developers/working-with-files-and-the-file-manager/overview
