Skip to main content

CLI Batch Download

The CLI allows you to download comments and predictions in batch. This is most useful to script import into analytics tools that don't require a live connection.

Which model version does the CLI tool use to get predictions?

The CLI tool will download the latest available computed predictions, i.e. the same predictions that are shown in the UI.

If you run the CLI after a new model has been trained, but before predictions for it had finished computing, you will get a mix of predictions from the latest model and the model before it. You can check whether predictions had finished computing in the dataset status in the UI.

Can I pass a specific model version to the CLI?

If you need predictions from a specific model version, consider using the Trigger and Predict API routes.

Setup#

Download the CLI tool for your platform (the CLI tool is available for Linux, Mac, and Windows).

Configure the command line tool:

re config add --name reinfer --endpoint https://reinfer.io/

You will be asked to paste your API token:

I A new context `reinfer` will be created.* Enter API token [none]: MYSUPERSECRETTOKENW Be careful, API tokens are stored in cleartext in /home/username/.config/reinfer/contexts.json.I New context `reinfer` was created.

Downloading Data#

The command below will download all comments and predictions in the specified source and dataset. Note that the dataset name and source name have to be prefixed by the name of the organisation they are in. If the dataset contains multiple sources, you need to issue this command for every source to download all comments in the dataset.

re get comments \  <org_name/source_name> \  --dataset <org_name/dataset_name> \  --predictions true \  --file <output_file_name.jsonl>

Processing Data#

info

Same as the API, the CLI tool returns predicted labels with confidence scores. In order to correctly process the confidence scores, be sure to check the Using Labels in Automation and Using Labels in Analytics sections of the Labels documentation.

The CLI returns data in JSONL format (also called newline-delimited JSON), where each line is a JSON value. Many tools will be able to process JSONL files out-of-the-box. Please let us know at support@reinfer.io if you have any questions.

Each line in the JSONL file will have the following format:

{  "comment": {...},  "labelling": {    "assigned": [...]    "predicted": [...]  },  "entities": {    "assigned": [...]    "predicted": [...]  }}
Field nameDescription
commentComment object in the format described here.
labelling.assignedList of assigned labels, in the format described here.
entities.assignedList of assigned entities, in the format described here.
labelling.predictedList of predicted labels, in the format described here.
entities.predictedList of predicted entities, in the format described here.

Note that the labelling or entities field may be absent completely if the comment has neither assigned nor predicted labels or entities.

Below is an example comment with predictions downloaded from a real-life dataset.

{  "comment": {    "id": "1234abcd",    "uid": "5678ef.1234abdc",    "timestamp": "2021-02-01T00:00:00Z",    "messages": [      {        "body": {          "text": "The hot chocolate biscuit on arrival raised my expectations"        }      }    ],    "user_properties": {      "string:Question": "What did you like about your stay",      "number:Reviewer Score": 5.4,      "number:Average Score": 8.4,      "number:Reviewer Total Number Of Reviews": 1,      "string:Hotel Name": "DoubleTree by Hilton London Victoria"    },    "created_at": "2021-02-01T00:00:00Z"  },  "labelling": {    "predicted": [      {        "name": "Refreshments",        "sentiment": 0.3598046874571062,        "probability": 0.54764723591506481      },      {        "name": "Property",        "sentiment": 0.6684685489411859,        "probability": 0.417815982922911644      }    ]  }}