DATA SOURCES : BEST PRACTICES

Brainwords

Data Sources
A type of flat file (csv, txt)
Start with the same name, with the same format, and contain the same tag list with a unique index

Data Properties
Excel file describing your variables

File specifications

Files names: name_of_the_file_YYYYMMDD_HHMMSS

Files formats: file .csv with delimiters such as « ; » or in .txt

Example: reporting_shift_20110525_134520.csv for the reporting_shift file of the 25th of May 2011 at 13:45:20

The data frequency of extraction must be defined. High frequency (ie: minute) allows the use of Braincube Live.

For example, high frequency registration is the process data and low frequency and/or non regular registration is Quality data, Event data, Weather data, Genealogy data, Organization data or Traceability data.

File structure and contents

Matrix Files

  • small files
  • extraction might be more complicated

List Files

  • easy extraction
  • large files

Mandatory

File name

  • Never change the file name or format
  • No special characters or spaces in file names

File Format

  • All files must have a unique index (timestamp, lot number…)
  • One file per type of data (process data, events data, …)
    ⇒ 1 tag per file is forbidden
  • If the delimiter is a semicolon ( ; ) please remove any semicolons in your text cells
  • Each type of file always contains the same tags
  • Never change the date format in the files

Data

  • You need to know if any filter or interpolation are applied to the values before extraction,
  • Send files with recovery when all data didn’t arrive at the same time (data will be overwritten by the last received),
  • Validate automatic extraction over a short period of time with Braincube before running the historical extraction.

Was this article helpful?

Powered by Zendesk