Difference between revisions of "Importing Text Data"

Revision as of 08:44, 14 September 2015

ARK textfile importer user guide

Since v1.1.2 ARK ahs had a tool to import text based files. in either JSON or CSV Data represented in either json or csv format can be uploaded to the ARK database.The file can be filtered so that only certain records are uploaded. It can create new records or add to existing ones.

Step 1. Identify datafile

Specify the location of the file.The import page of ARK in 1.1.2 includes a function for importing data directly from a text file or a data stream available online. Currently the file must be available on a web server. (Fig. 1) Enter the location of the file, either a relative path from the ARK root eg data/uploads/your_data.csv or a remote web address eg http://www.example.com/ARK/data?format=json. Click Submit.

Depend on the size of the file it may take some time to load into your browser. Once it is loaded it will be available to manipulate very quickly.A spinner will appear while this is happening.

Step 2. Determine 'Root' of the data structure.

The data you have will likely include data that relates to several ARK items.The level that these repeat in the data structure must be defined so that ARK can import all the records automatically. In a csv file where there is a single record per row the root will be 'root>items'. In a more complicated json file (for example one with file level metadata held in the root as well as the items) the records may be contained in a object within the root object.

In order to define this level you will need to navigate to it.The path at the top of the page describes where in the data structure you currently are.The large panel below the advanced options shows the identifiers held in the current level.At the root level of a CSV file the first column will be used as an identifier. Clicking on these will open the data attached to that identifier for viewing.(Fig. 2) It is possible to go up one level using the button at the end of the list of objects, or any part of the location at the top of the page can be used to navigate to that level.When the main window would contain more than 20 items it is truncated and only the first 20 are shown.

Figure 1: File Picker

Use the 'this is root' button in the navigation panel at the top of the page when the root is displayed in the main panel.

Figure 2: Objects in 'root>items'

Step 3. Determine ARK IDs

Using the navigation method above find the path to the ARK id within your data.This will often be as simple as 'root>items>item1>Ark_id'.The object that you define the ARK ID in is not important.As long as the objects are structured consistently the ARK IDs will be retrieved automatically based on the root defined in the step above.

Figure 3: Root and ARK ID specified If you do not have ARK IDs in your data refer to the advanced options. Otherwise, click 'This is ARK ID' button below the identifier that contains the ARK ID.

Step 4. Test Import

Click 'Import this' below the identifier that you would like to import. If you have completed the steps above a new panel will appear below the viewer. Confirm that this looks how you expect – this is what will be imported into the database.

The final panel allows you to define which records will be imported.A type must be specified. If you are importing a file it may be necessary to do a separate import of each type. This is because different types have different fields attached to them, the drop down menus respond to the ones above them, presenting only the options available. If they are not completed in order it may result in unexpected behaviour.

Step 5.

When you are happy that you are importing the correct data into the correct field, click 'Submit'. You will be shown the results of your import. Click 'Import Json' to repeat the process for any other fields you wish to import.

Advanced Options

Ste_cd:The site code that imported items will have, is one is not specified in the data – uses the ARK default if not set. No site code: This check box will remove site codes from the import itemvalue, for use with chains. Regular Expression: This is used to extract the itemvalue from the ARK ID column – by default it grabs the first run of numbers in the field. Start at Arbitrary Number: If you wish to create a sequence of numbers starting from a given point, (including 1) you will need to specify this here, and the number to start on. A unique 'ARK ID' must still be specified in the data, but it will not be used.