Flow Metrics API

Table of Contents

Use the Flow Metrics API to get metrics describing how a flow performed over a period of time. Instabase can track review metrics for all flow jobs that have a pipeline attached when the job is kicked off. Developers can use this API to pull reviewer efficiency data from Instabase and process elsewhere to monitor the overall efficiency of reviewer teams.

Aggregated flow review metrics

Method	Syntax
POST	`URL_BASE/flow_metrics`

Description

Get an overview of aggregated job reviewer metrics given a certain set of filters. The request must be a JSON body including the parameters listed below. Metrics are only counted for jobs in which a review event occurred, which requires jobs to have checkpoints that trigger human review through Flow Review.

Metrics definitions

total_jobs_needs_review
- The number of jobs which had a review event awaiting review at end_time.
total_jobs_in_review
- The number of jobs which had a review event in review at end_time.
total_jobs_completed
- The number of jobs which completed within the specified time range.
total_jobs
- The sum of total_jobs_needs_review, total_jobs_in_review, and total_jobs_completed
avg_duration_awaiting_review
- The average time between review event creation and its reviewer being assigned. This time metric takes into account all jobs that had a review event marked for review within the specified time range.
avg_duration_in_review
- The average time between a review event having a reviewer assigned and the reviewer marking the batch as reviewed. This time metric takes into account all jobs that had a review event complete within the specified time range.

Request parameters

Parameters are required unless marked as optional.

Name	Type	Description	Values
`filter_by`	object	Optional. Filters to limit jobs being aggregated	see below for filter keys
`filter_by/pipline_id`	Tuple	Optional. Filter jobs that belong to a pipeline, such as `("895dfb05-81a7-4725-a9aa-4be41377484d", "MATCH")`	(pipeline id, “MATCH” or “WILDCARD”)
`filter_by/name`	Tuple	Optional. Filter jobs run by a particular marketplace solution, such as `("bank_statements", "MATCH")`	(any marketplace solution name, “MATCH” or “WILDCARD”)
`filter_by/version`	Tuple	Optional. Filter jobs run with a particular version of specified marketplace solution, such as `("v3.1.4", "MATCH")`	(marketplace solution version, “MATCH” or “WILDCARD”)
`metrics`	array	Optional. a list of metrics to return, if none passed all metrics are returned	`total_jobs_needs_review`, `total_jobs_in_review`, `total_jobs_completed`, `total_jobs`, `avg_duration_awaiting_review`, `avg_duration_in_review`
`start_time`	integer	Lower bound of the time range to restrict results, in milliseconds since epoch.	milliseconds since epoch
`end_time`	integer	Upper bound of the time range to restrict results, in milliseconds since epoch.	milliseconds since epoch

Response schema

Key	Description	Value
`status`	Status of response	`OK`,`ERROR`
`data/results`	Main response body	see below for keys
`data/results/metric_name`	Value object for one of the requested metrics	`total_jobs_needs_review`, `total_jobs_in_review`, `total_jobs_completed`, `total_jobs`, `avg_duration_awaiting_review`, `avg_duration_in_review`
`data/results/metric_name/count`	Value of metric	integer
`data/results/metric_name/start_time`	Start time of filter	milliseconds since epoch
`data/results/metric_name/end_time`	End time of filter	milliseconds since epoch

Examples

Request

POST URL_BASE/flow_metrics

# body
{
  "filter_by": {
    "pipeline_id": ("895dfb05-81a7-4725-a9aa-4be41377484d", "MATCH"),
  },
  "metrics": ["total_jobs_needs_review", "total_jobs"]
  "start_time": 1658966399999,
  "end_time": 1658805245429
}

Response

HTTP STATUS CODE 200

# body
{
  "status": "OK",
  "data": {
    "results": {
        "total_jobs_needs_review": {
            "count": 3,
            "start_time": 1658966399999,
            "end_time": 1658805245429
        },
        "total_jobs": {
            "count": 10,
            "start_time": 1658966399999,
            "end_time": 1658805245429
        },
        ...
    }
  }
}

Jobs Table Query

Method	Syntax
POST	`URL_BASE/flow_metrics/jobs`

Description

Returns a table where rows represent jobs and columns represent various metrics describing how the job progressed through human review. Only jobs that had a review event that started within the time range are included.

Request parameters

POST body parameters

Name	Type	Description	Values
`start_time`	integer	Lower bound of the time range to restrict results, in milliseconds since epoch.	milliseconds since epoch
`end_time`	integer	Upper bound of the time range to restrict results, in milliseconds since epoch.	milliseconds since epoch
`filter_by`	object	Optional. Filters to limit jobs being aggregated	see below for filter keys
`filter_by/pipline_id`	string	Optional. Filter jobs that belong to a pipeline	uuid
`filter_by/name`	string	Optional. Filter jobs that were run with a particular marketplace solution	any marketplace solution name
`filter_by/version`	string	Optional. Filter jobs run with a particular version of specified marketplace solution	marketplace solution version
`size`	integer	Optional. Number of results to return	(default 20)
`offset`	integer	Optional. N-th entry to return	(default 0)

Size and offset are used to paginate the results because query performance suffers as the size increases.

Query string parameters

Name	Type	Description	Values
`csv`	boolean	return the response as csv	(default: `false`), `true`
`pretty`	boolean	prettify the return json	(default: `false`), `true`

Response schema

The resulting list of jobs is sorted by earliest_review_timestamp in ascending order. The earliest_review_timestamp means the earliest timestamp (within the time range) of review event creation for the given job ID.

Key	Description	Value
`status`	Status of response	`OK`,`ERROR`
`data/results`	Array of jobs	See below for all keys
`data/results/job_id`	UUID of the job	uuid
`data/results/job_start_time`	Time when job started	milliseconds since epoch
`data/results/earliest_review_timestamp`	Time when reviewer started review	milliseconds since epoch or null
`data/results/duration_awaiting_review`	Time between job finishing and reviewer starting their review. If review has not started, this is null.	milliseconds or null
`data/results/duration_in_review`	Time between reviewer starting review and finishing review. If review has not finished, this is null.	milliseconds or null
`data/results/job_finish_time`	Time when job finished being reviewed. If not finished, this is null.	milliseconds since epoch or null

Examples

Request

POST URL_BASE/flow_metrics/jobs

# body
{
  "filter_by":
    {
      "pipeline_id":["ae026563-09a2-4dc5-989d-27ea755de67a"]
    },
  "start_time":"1656300166379",
  "end_time":"1658966399999",
  "offset":0,
  "size":20,
}

Response

HTTP STATUS CODE 200

# body
{
  "status": "OK",
  "data": {
    "results": [
      {
        "job_id": "a8c5cda1-b04a-4b40-997a-a8d8db2edde6",
        "job_start_time": 1657326845505,
        "job_finish_time": null,
        "earliest_review_timestamp": 1657326859099.0,
        "duration_awaiting_review": 0.0,
        "duration_in_review": 0.0
      }
    ]
  }
}

Request: CSV format

POST URL_BASE/flow_metrics/jobs?csv=true

# body
{
  "filter_by":
    {
      "pipeline_id":["ae026563-09a2-4dc5-989d-27ea755de67a"]
    },
  "start_time":"1656300166379",
  "end_time":"1658966399999",
  "offset":0,
  "size":20,
}

Response

job_id	job_start_time	job_finish_time	duration_awaiting_review	duration_in_review	earliest_review_timestamp
“a8c5cda1-b04a-4b40-997a-a8d8db2edde6”	1657326845505	null	1657326859099	0	0

Flow Review Log Query

Method	Syntax
GET	`URL_BASE/flow_metrics/flow_review_log/{job_id}`

Description

Get a table where rows represent review events tagged to the specified job_id and values represent various metrics describing how an individual review event was dealt with. Only review events eclipsing the time range are included.

Request parameters

POST body parameters

Name	Type	Description	Values
`start_time`	integer	Lower bound of the time range to restrict results, in milliseconds since epoch.	milliseconds since epoch
`end_time`	integer	Upper bound of the time range to restrict results, in milliseconds since epoch.	milliseconds since epoch
`filter_by`	object	Optional. Filters to limit jobs being aggregated	see below for filter keys
`filter_by/pipline_id`	string	Optional. Filter jobs that belong to a pipeline	uuid
`filter_by/name`	string	Optional. Filter jobs that were run with a particular marketplace solution	any marketplace solution name
`filter_by/version`	string	Optional. Filter jobs run with a particular version of specified marketplace solution	marketplace solution version
`size`	integer	Optional. Number of results to return	(default 20)
`offset`	integer	Optional. N-th entry to return	(default 0)

Tip

Query performance suffers as the number of returned results increases. To mitigate this impact, use the size and offset parameters to paginate results.

Query string parameters

Name	Type	Description	Values
`csv`	boolean	return the response as csv	(default: `false`), `true`
`pretty`	boolean	prettify the return json	(default: `false`), `true`

Response schema

The resulting list of jobs is sorted by time_failed_validation in ascending order. A job might have review events across various pipelines—this can lead to the query returning different results based on the value of pipeline_id in the filter_by clause.

Key	Description	Value
`status`	Status of response	`OK`,`ERROR`
`data/results`	Array of jobs	see below for job keys
`data/results/job_id`	UUID of the job	uuid
`data/results/review_id`	UUID of the review event	uuid
`data/results/pipeline_id`	UUID of the pipeline review event occurred	uuid
`data/results/time_failed_validation`	Time when job failed validation	milliseconds since epoch
`data/results/duration_awaiting_review`	Time between job finishing and reviewer starting their review. If review has not started, this is null.	milliseconds or null
`data/results/time_marked_reviewing`	Time when job was marked as in review. If review has not started, this is null.	milliseconds since epoch
`data/results/duration_in_review`	Time between reviewer starting review and finishing review. If review has not finished, this is null.	milliseconds or null
`data/results/time_completed_review`	Time when job finished being reviewed. If not finished, this is null.	milliseconds since epoch or null
`data/results/reviewer_username`	Username of the reviewer associated with the review event	string

Examples

Request

POST URL_BASE/flow_metrics/flow_review_log/{job_id}

# body
{
  "filter_by":
    {
      "pipeline_id":["ae026563-09a2-4dc5-989d-27ea755de67a"]
    },
  "start_time":"1656300166379",
  "end_time":"1658966399999",
  "offset":0,
  "size":20,
}

Response

HTTP STATUS CODE 200

# body
{
  "status": "OK",
  "data": {
    "results": [
        {
          "review_id": "1869f0cf-eecf-4fed-9541-3dd779931d53",
          "job_id": "f8571ab7-9971-424b-a762-6ada6e5ea9f1",
          "pipeline_id": "ae026563-09a2-4dc5-989d-27ea755de67a",
          "time_failed_validation": 1657121916701,
          "duration_awaiting_review": 7555,
          "time_marked_reviewing": 1657121924256,
          "duration_in_review": 8582,
          "time_completed_review": 1657121932838,
          "reviewer_username": "reviewer_user"
        },
    ]
  }
}

Request: CSV format

POST URL_BASE/flow_metrics/jobs?csv=true

# body
{
  "filter_by":
    {
      "pipeline_id":["ae026563-09a2-4dc5-989d-27ea755de67a"]
    },
  "start_time":"1656300166379",
  "end_time":"1658966399999",
  "offset":0,
  "size":20,
}

Response

review_id	job_id	pipeline_id	time_failed_validation	duration_awaiting_review	time_marked_reviewing	duration_in_review	time_completed_review	reviewer_username
“1869f0cf-eecf-4fed-9541-3dd779931d53”	“f8571ab7-9971-424b-a762-6ada6e5ea9f1”	“ae026563-09a2-4dc5-989d-27ea755de67a”	1657121916701	7555	1657121924256	8582	1657121932838	“reviewer_user”