Importing Text Data - Revision history

Mike: /* Step 1. Identify datafile */

2018-04-03T12:17:55Z

‎Step 1. Identify datafile

2018-04-03T12:15:57Z

‎ARK textfile importer user guide

2015-12-09T10:29:17Z

‎Step 3. Determine ARK IDs

2015-12-09T10:25:35Z

‎Step 2. Determine 'Root' of the data structure.

2015-11-17T08:57:07Z

‎Advanced Options

2015-11-17T08:50:48Z

2015-09-14T11:38:04Z

‎Step 4. Test Import

2015-09-14T11:34:45Z

‎Step 3. Determine ARK IDs

2015-09-14T11:34:18Z

‎Step 3. Determine ARK IDs

2015-09-14T11:32:10Z

‎Step 2. Determine 'Root' of the data structure.

@@ Line 5: / Line 5: @@
 ===Step 1. Identify datafile===
 [[File:jsonhowto_fig1.png|thumb|400px|right|Figure 1: File Picker]]
-Specify the location of the file.The import page of ARK in 1.1.2 includes a function for importing data directly from a text file or a data stream available online. Currently the file must be available on a web server. (Fig. 1) Enter the location of the file, either a relative path from the ARK root eg data/uploads/your_data.csv or a remote web address eg http://www.example.com/ARK/data?format=json.
+;Specify the location of the file.
 Click Submit.

@@ Line 1: / Line 1: @@
 ==ARK textfile importer user guide==
-Since v1.1.2 ARK ahs had a tool to import text based files. in either JSON or CSV
+Since v1.1.2 ARK has had a tool to import text based files in either JSON or CSV.
-Data represented in either json or csv format can be uploaded to the ARK database.The file can be filtered so that only certain records are uploaded. It can create new records or add to existing ones.
+.The file can be filtered so that only certain records are uploaded. It can create new records or add to existing ones.
 ===Step 1. Identify datafile===

← Older revision		Revision as of 10:29, 9 December 2015
Line 29:		Line 29:


−	If you do not have ARK IDs in your data refer to the advanced options.	+	If you do not have ARK IDs in your data refer to the [[#Advanced Options\|advanced options]].
	Otherwise, click 'This is ARK ID' button below the identifier that contains the ARK ID.		Otherwise, click 'This is ARK ID' button below the identifier that contains the ARK ID.

@@ Line 14: / Line 14: @@
 The data you have will likely include information that relates to several ARK items. Within your data structure there will be a level that includes all of these item representations - in the same way as rows in a table. In a csv file where there is a single record per row the root will be 'root>items'. In a more complicated json file (for example one with file level metadata held in the root as well as the items) the records may be contained in an object within the root object. Json allows for extensible representation, so each 'row' in the json file may not include all the same fields. It may be necessary to use the filter tools to only import objects which have a certain criteria.
-The example shown here has many items in the items object in root. The first available row is used to identify each of the contained objects - in this case it is an ARK ID, but it could be anything and may not be unique if it is not unique in your structure. For example if field 1 in your table is a site code or context type.
+The example shown here has many items in the items object in root. The first available column is used to identify each of the contained objects - in this case it is an ARK ID, but it could be anything and may not be unique if it is not unique in your structure. For example if field 1 in your table is a site code or context type.
 [[File:jsonhowto_fig2.png|thumb|400px|left|none|Figure 2: Objects in 'root>items']]

@@ Line 45: / Line 45: @@
 ==Advanced Options==
-#Ste_cd:The site code that imported items will have, is one is not specified in the data – uses the ARK default if not set. No site code: This check box will remove site codes from the import itemvalue, for use with chains.
+;Ste_cd:
-#Regular Expression: This is used to extract the itemvalue from the ARK ID column – by default it grabs the first run of numbers in the field.
+:The site code that imported items will have, is one is not specified in the data – uses the ARK default if not set. No site code: This check box will remove site codes from the import itemvalue, for use with chains.
-#Start at Arbitrary Number: If you wish to create a sequence of numbers starting from a given point, (including 1) you will need to specify this here, and the number to start on. A unique 'ARK ID' must still be specified in the data, but it will not be used.
+;Regular Expression:

@@ Line 34: / Line 34: @@
 ===Step 4. Test Import===
-The final panel allows you to define which records will be imported.A type must be specified. If you are importing a file it may be necessary to do a separate import of each type. This is because different types have different fields attached to them, the drop down menus respond to the ones above them, presenting only the options available. If they are not completed in order it may result in unexpected behaviour.
+[[File:jsonhowto_fig4.png|thumb|right|400px|Figure 4: Example of dry run]]
 ===Step 5.===

@@ Line 24: / Line 24: @@
 ===Step 3. Determine ARK IDs===
 Using the navigation method above find the path to the ARK id within your data. This will often be as simple as 'root>items>item1>Ark_id'.The object that you define the ARK ID in is not important. As long as the objects are structured consistently the ARK IDs will be retrieved automatically based on the root defined in the step above. so for instance the importer will import the ARK ID for METSUR_152 from root > items > METSUR_152 > ARK_id and then the ARK ID for METSUR_153 from root > items > METSUR_153 > ARK_id, and so on.
 If you do not have ARK IDs in your data refer to the advanced options.