Overview

As described in KPI Concepts , there are two steps to managing KPIs in Ascerta. The first is to define the KPI for a use case, and the second is to provide the scores, or data, for those KPIs.

You can accomplish both steps either in the Ascerta UI or via the SDK. However, since data entry for the KPIs would be extremely manual, it is highly recommended to set the KPI scores in your application's code.

🕹️Managing Use Cases and KPIs in the UI

The following interactive tutorial walks through how to manage use cases and KPIs in the Ascerta UI.

For the best experience, maximize the interactive tutorial and turn on audio.

Managing KPIs in your Application Code

The SDK provides methods for:

Creating KPI definitions for use case types
Recording KPI values for specific use case instances
Retrieving, updating, and deleting KPI data

As with all Ascerta types, KPI definition is idempotent, and repeated creation calls will not result in failures or duplication.

from ascerta import Ascerta

client = Ascerta()

# Step 1: Define a KPI for a use case type
kpi = client.use_cases.definitions.kpis.create(
    use_case_name="Chat-Bot",
    kpi_name="Deflection Rate",
    description="Tracks when AI resolves issues without human intervention",
    kpi_type="boolean",
    goal=0.25
)

# Step 2: Record a score for a specific instance
client.use_cases.kpis.update(
    use_case_name="Chat-Bot",
    kpi_id=kpi.kpi_id,
    use_case_id="uc_123456789",
    score=True
)

Use Case Instance IDs

KPI scores are always associated with a specific use case instance, identified by a use_case_id. This connection ensures that your custom metrics are linked to the exact context in which they were generated, providing a comprehensive view alongside the standard metrics Ascerta already tracks:

client.use_cases.kpis.update(
    use_case_name="Document-Summarizer",
    kpi_name="Customer Satisfaction", # Likert5 KPI
    use_case_id=use_case_id, # ID of the instance to set the score
    score=4 # Rating on 1-5 scale
)

If you did not define the use_case_id when creating the instance, then Ascerta will automatically assign one for you. The generated use_case_id can be retrieved using get_context().

from ascerta import get_context

def print_use_case_info():
  context = get_context()
    
  if context.get("use_case_name", ''):
      print(
          f"Current use case: {context.get('use_case_name')} "
          f"(ID: {context.get('use_case_id', '')}, "
          f"Version: {context.get('use_case_version', '')})"
      )

      if context.get("use_case_step"):
          print(f"Current step: {context.get('use_case_step')}")
  else:
      print("No use case active in current context")

KPI Types and Scores

The scores for each KPI Type must be represented in the following value types.

KPI Type	Score Value Type
`boolean`	Boolean or Integer (0,1)
`number`	float
`percentage`	float
`likert5`	Integer (1-5)
`likert7`	Integer (1-7)
`likert10`	Integer (1-10)

Incrementing KPI number scores

If the KPI number score value is not stored externally, the last score KPI score value set in Ascerta can be used as the base value. Two helper functions are provided: increment_kpi_score and increment_kpi_score_async

from ascerta.lib.helpers import increment_kpi_score_async, increment_kpi_score

Parameter	Type	Required
`ascerta`	`Ascerta` or `AsyncAscerta`	Yes
`kpi_id`	str	Yes
`kpi_name`	str	No
`use_case_name`	str	Yes
`use_case_id`	str	Yes
`increment`	int	No, defaults to 1

from ascerta import Ascerta
from ascerta.lib.helpers import increment_kpi_score
# from ascerta.lib.helpers import increment_kpi_score_async

client = Ascerta()

kpi = client.use_cases.definitions.kpis.create(
    use_case_name="document_generator",
    kpi_name="Rejected Pages",
    description="Tracks number of rejected pages",
    kpi_type="number",
    goal=0
)

# Agent generates additional pages and now 1 is rejected in addition to previous rejections
increment_kpi_score(
    ascerta=client,
    use_case_name="document_generator",
    kpi_id=kpi.kpi_id,
    kpi_name="Rejected Pages",
    use_case_id="uc_123456789",
    score=1
)

Ingesting KPI scores from other systems

Your KPI data may live in a variety of systems that live apart from your GenAI code or the Ascerta library. The API to set KPI scores is available via a standard REST interface so that it can be called from virtually anywhere and from any programming language, without the need for the Ascerta SDK.

Note that the use_case_name, use_case_id, and kpi_id are represented in the URL path.

curl --request PUT \
     --url "https://<YOUR ASCERTA HOST>/api/v1/use_cases/instances/{use_case_name}/{use_case_id}/kpis/{kpi_id}" \
     --header 'accept: application/json' \
     --header 'content-type: application/*+json' \
     --header 'xProxy-api-key: your_ASCERTA_API_KEY' \
     --data '{"score":123}'