tensorone logo

SDKs

Javascript Endpoints

Interacting with Tensor One's endpoints is a core feature of the SDK, enabling the execution of tasks and the retrieval of results. This section covers the synchronous and asynchronous execution methods, along with checking the status of operations.

Prerequisites

Before using the Tensor One Python, ensure that you have:

  • Installed the Tensor One Python SDK.
  • Configured your API key.

Set your Endpoint Id

Set your Tensor One API key and your Endpoint Id as environment variables.

import tensoroneSdk from 'tensorone-sdk'

const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env
const tensorone = tensoroneSdk(TENSORONE_API_KEY)
const endpoint = tensorone.endpoint(ENDPOINT_ID)

This allows all calls to pass through your Endpoint Id with a valid API key.

In most situations, you'll set a variable name endpoint on the Endpoint class. This allows you to use the following methods or instances variables from the Endpoint class:

  • health
  • purge_queue
  • runSync
  • run

Run the Endpoint

Run the Endpoint with the either the asynchronous run or synchronous runSync method.

Choosing between asynchronous and synchronous execution hinges on your task's needs and application design.

Run synchronously

To execute an endpoint synchronously and wait for the result, use the runSync method on your endpoint. This method blocks the execution until the endpoint run is complete or until it times out.

import tensoroneSdk from 'tensorone-sdk'

const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env
const tensorone = tensoroneSdk(TENSORONE_API_KEY)
const endpoint = tensorone.endpoint(ENDPOINT_ID)

const result = await endpoint.runSync({
  input: {
    prompt: 'Hello, World!',
  },
})

console.log(result)

Run asynchronously

Asynchronous execution allows for non-blocking operations, enabling your code to perform other tasks while waiting for an operation to complete.

For non-blocking operations, use the run method on the endpoint. This method allows you to start an endpoint run and then check its status or wait for its completion at a later time.

import tensoroneSdk from 'tensorone-sdk'

const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env
const tensorone = tensoroneSdk(TENSORONE_API_KEY)
const endpoint = tensorone.endpoint(ENDPOINT_ID)

const result = await endpoint.run({
  input: {
    prompt: 'Hello, World!',
  },
})

console.log(result)

Get results from an asynchronous run

The following example shows how to get the results of an asynchronous run.

import tensoroneSdk from 'tensorone-sdk'

const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms))

async function main() {
  const tensorone = tensoroneSdk(TENSORONE_API_KEY)
  const endpoint = tensorone.endpoint(ENDPOINT_ID)
  const result = await endpoint.run({
    input: {
      prompt: 'Hello, World!',
    },
  })

  console.log(result)
  console.log('run response')
  console.log(result)

  const { id } = result // Extracting the operation ID from the initial

  // Check the status in a loop, similar to the working example
  for (let i = 0; i < 20; i++) {
    // Increase or decrease the loop count as necessary
    const statusResult = await endpoint.status(id)
    console.log('status response')
    console.log(statusResult)

    if (
      statusResult.status === 'COMPLETED' ||
      statusResult.status === 'FAILED'
    ) {
      // Once completed or failed, log the final status and break the loop
      if (statusResult.status === 'COMPLETED') {
        console.log('Operation completed successfully.')
        console.log(statusResult.output)
      } else {
        console.log('Operation failed.')
        console.log(statusResult)
      }
      break
    }
    // Wait for a bit before checking the status again
    await sleep(5000)
  }
}

main()

Poll the status of an asynchronous run

Uses await endpoint.status(id) to check the status of the operation repeatedly until it either completes or fails. After each check, the function waits for 5 seconds (or any other suitable duration you choose) before checking the status again, using the sleep function. This approach ensures your application remains responsive and doesn't overwhelm the Tensor One endpoint with status requests.

import tensoroneSdk from 'tensorone-sdk'

const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env
// Function to pause execution for a specified time
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms))

async function main() {
  try {
    const tensorone = tensoroneSdk(TENSORONE_API_KEY)
    const endpoint = tensorone.endpoint(ENDPOINT_ID)
    const result = await endpoint.run({
      input: {
        prompt: 'Hello, World!',
      },
    })

    const { id } = result

    if (!id) {
      console.error('No ID returned from endpoint.run')
      return
    }

    // Poll the status of the operation until it completes or fails
    let isComplete = false

    while (!isComplete) {
      const status = await endpoint.status(id)
      console.log(`Current status: ${status.status}`)

      if (status.status === 'COMPLETED' || status.status === 'FAILED') {
        isComplete = true // Exit the loop
        console.log(`Operation ${status.status.toLowerCase()}.`)

        if (status.status === 'COMPLETED') {
          console.log('Output:', status.output)
        } else {
          console.error('Error details:', status.error)
        }
      } else {
        await sleep(5000) // Adjust the delay as needed
      }
    }
  } catch (error) {
    console.error('An error occurred:', error)
  }
}

main()

Stream

Stream allows you to stream the output of an Endpoint run. To enable streaming, your handler must support the "return_aggregate_stream": True option on the start method of your Handler. Once enabled, use the stream method to receive data as it becomes available.

import tensoroneSdk from 'tensorone-sdk'
const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env

async function main() {
  const tensorone = tensoroneSdk(TENSORONE_API_KEY)
  const endpoint = tensorone.endpoint(ENDPOINT_ID)
  const result = await endpoint.run({
    input: {
      prompt: 'Hello, World!',
    },
  })
  console.log(result)
  const { id } = result
  for await (const result of endpoint.stream(id)) {
    console.log(`${JSON.stringify(result, null, 2)}`)
  }
  console.log('done streaming')
}
main()

Health check

Monitor the health of an endpoint by checking its status, including jobs completed, failed, in progress, in queue, and retried, as well as the status of workers.

import tensoroneSdk from 'tensorone-sdk'

const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env
const tensorone = tensoroneSdk(TENSORONE_API_KEY)
const endpoint = tensorone.endpoint(ENDPOINT_ID)
const health = await endpoint.health()

console.log(health)

Status

Use the status method and specify the id of the run to get the status of a run.

import tensoroneSdk from 'tensorone-sdk'
const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env

async function main() {
  try {
    const tensorone = tensoroneSdk(TENSORONE_API_KEY)
    const endpoint = tensorone.endpoint(ENDPOINT_ID)
    const result = await endpoint.run({
      input: {
        prompt: 'Hello, World!',
      },
    })

    const { id } = result
    if (!id) {
      console.error('No ID returned from endpoint.run')
      return
    }

    const status = await endpoint.status(id)
    console.log(status)
  } catch (error) {
    console.error('An error occurred:', error)
  }
}

main()

Cancel

You can cancel a Job request by using the cancel() function on the run request. You might want to cancel a Job because it's stuck with a status of IN_QUEUE or IN_PROGRESS, or because you no longer need the result.

import tensoroneSdk from 'tensorone-sdk'
const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env

async function main() {
  try {
    const tensorone = tensoroneSdk(TENSORONE_API_KEY)
    const endpoint = tensorone.endpoint(ENDPOINT_ID)
    const result = await endpoint.run({
      input: {
        prompt: 'Hello, World!',
      },
    })
    const { id } = result

    if (!id) {
      console.error('No ID returned from endpoint.run')
      return
    }

    const cancel = await endpoint.cancel(id)
    console.log(cancel)
  } catch (error) {
    console.error('An error occurred:', error)
  }
}

main()

Timeout

To set a timeout on a run, pass a timeout value to the run method. Time is measured in milliseconds.

import tensoroneSdk from 'tensorone-sdk'

const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env
const tensorone = tensoroneSdk(TENSORONE_API_KEY)
const endpoint = tensorone.endpoint(ENDPOINT_ID)
const result = await endpoint.run(
  {
    input: {
      prompt: 'Hello, World!',
    },
  },
  5000
)

console.log(result)

Execution policy

You can set the maximum time to wait for a response from the endpoint in the policy parameter.

import tensoroneSdk from 'tensorone-sdk'

const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env
const tensorone = tensoroneSdk(TENSORONE_API_KEY)
const endpoint = tensorone.endpoint(ENDPOINT_ID)
const result = await endpoint.run({
  input: {
    prompt: 'Hello, World!',
  },
  policy: {
    executionTimeout: 5000,
  },
})

console.log(result)

Purge queue

You can purge all jobs from a queue by using the purgeQueue() function.

purgeQueue() doesn't affect Jobs in progress.

import tensoroneSdk from 'tensorone-sdk'
const { TENSORONE_API_KEY, ENDPOINT_ID } = process.env

async function main() {
  try {
    const tensorone = tensoroneSdk(TENSORONE_API_KEY)
    const endpoint = tensorone.endpoint(ENDPOINT_ID)
    await endpoint.run({
      input: {
        prompt: 'Hello, World!',
      },
    })

    const purgeQueue = await endpoint.purgeQueue()
    console.log(purgeQueue)
  } catch (error) {
    console.error('An error occurred:', error)
  }
}
main()
Previous
Javascript
Next
Python