To measure the performance of a model on a dataset, it can be done by comparing the model's predictions to some ground truth references. We currently offer BLEU, ROUGE, WER, METEOR, and Accuracy.

Create Dataset Schema

Access Datasets

• Navigate to the 'Datasets' section from the left panel.

Create Dataset

• Click on 'Create dataset'.

• Provide the following information:

  • Name: Give your dataset a unique name.

  • Description: Describe the dataset for future reference.

  • Evaluation Variables: This will be your data schema and variables used for evaluators. To evaluate the dataset as a whole, select "Prediction, Reference" from the dropdown menu.

• Click on 'Submit' to save the dataset schema.

Populate Dataset

Select the Dataset to Populate

• Choose the dataset you'd like to populate from your list of datasets.

Import Data

• Click the 'Import' button at the top right corner.

• Upload your CSV file.

• Map your desired columns from the CSV file to the evaluation variables.

• Note that the maximum number of rows is currently capped at 1000. If you need extra rows, please contact us.

Other Methods to Populate Dataset

• You can also create individual data points manually, save data from the playground, or inferences.

Evaluate Dataset

Evaluate Dataset

• Click on 'Evaluate dataset'.

• Choose the desired dataset evaluators from the list.

Review Evaluator Description

• Upon selection, a description of the evaluator will appear. Review this to ensure it matches your needs.

• If everything looks right, click 'Create'.

Need Help?

If you have any questions or need assistance, feel free to contact us at team@ownlayer.com or schedule a call with us here.