Skip to main content
Version: mainnet (v0.71)

Using data sources

Choosing a data source when proposing a market​

A market proposal must specify details about the data it requires in the market creation governance proposal. When configuring a market's instrument, you will need to select the data sourcefor two events: settlement and trading termination.

This is done by:

  1. Defining a data source spec binding for settlement price
  2. Configuring a data source spec for settlement price values
  3. Defining a data source spec binding for trading termination
  4. Configuring a data source spec for trading termination values

The binding tells the market which field contains the value. The spec defines which public keys to watch for data from, and which values to pass through to the binding.

When it's time for a market to settle, someone needs to submit the data that matches the data source spec defined in the market.

Read more:

Market governance concepts: Tutorial - proposing a market:

Who can submit data​

Any Vega keypair can submit data. In the configuration for a market, a data source specification field dictates which data feeds it is interested in. In effect, it works as a filter. This specification means that the creator of an instrument for a market will choose in advance a price source, and which data fields the market requires to settle and terminate.

Open Oracle signed messages​

Vega's Data Sourcing framework supports signed ABI-encoded Open Oracle ↗ or JSON messages. ABI-encoded signed messages can be verified to have come from the public key that signed them, which allows markets on Vega to use pricing data sourced from Ethereum.

Using Open Oracle signed messages in a market proposal​

For the binding, use the name field of the data. In the case of Open Oracle messages, the price data will be availableas 'prices.currency-code.value', for example:"prices.BTC.value".

For now this will focus on using the data for settlement price - both examples below use a Vega time data source to terminate the market.

"dataSourceSpecBinding": {
  "settlementDataProperty": "prices.BTC.value",
  "tradingTerminationProperty": "vegaprotocol.builtin.timestamp"
}

The following spec would make the market use the BTC value from the Coinbase Price Oracle ↗ data that is submitted in a subsequent example:

   "dataSourceSpecForSettlementData": {
        "signers": [{"ethAddress": { "address": "0xfCEAdAFab14d46e20144F48824d0C09B1a03F2BC" } }],
        "filters": [{
            "key": {
                "name": "prices.BTC.timestamp",
                "type": "TYPE_INTEGER",
            },
            "conditions": [{
                "operator": "OPERATOR_GREATER_THAN",
                "value": "1649265840",
            }]
        }]
    }

The signers: ethAddress in this case is the Ethereum public key that signed the data in the message.

Submitting Open Oracle data​

Use the command line to submit an Open Oracle message as a transaction that is signed by your Vega wallet and sent to the validators for consensus.

Below, find instructions on how to submit Open Oracle data as a signed message. Markets should be configured to only use the data at the relevant time, such as after a defined settlement date, in the market proposal.

1. Obtain an Open Oracle message​

{
    "timestamp": "1649265840",
    "messages": ["0x000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000624dccb000000000000000000000000000000000000000000000000000000000000000c00000000000000000000000000000000000000000000000000000000a2e04f5f00000000000000000000000000000000000000000000000000000000000000006707269636573000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000034254430000000000000000000000000000000000000000000000000000000000"],
    "signatures": ["0x8362a456997287a6b89e2de52e26c2aca423ab0ed401f9a23c81da2e2c56a5db27365adcb478d7b36558df58ca5dd240191a0f08a7f0ed79ee23cec77521e5c2000000000000000000000000000000000000000000000000000000000000001b"],
    "prices": {
        "BTC": "43721.75"
    }
}

2. Encode the Open Oracle message​

  echo '{"timestamp":"1649265840","messages":["0x000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000624dccb000000000000000000000000000000000000000000000000000000000000000c00000000000000000000000000000000000000000000000000000000a2e04f5f00000000000000000000000000000000000000000000000000000000000000006707269636573000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000034254430000000000000000000000000000000000000000000000000000000000"],"signatures":["0x8362a456997287a6b89e2de52e26c2aca423ab0ed401f9a23c81da2e2c56a5db27365adcb478d7b36558df58ca5dd240191a0f08a7f0ed79ee23cec77521e5c2000000000000000000000000000000000000000000000000000000000000001b"],"prices":{"BTC":"43721.75"}}' | base64

It will return a payload string that will look something like this:

eyJ0aW1lc3RhbXAiOiIxNjQ5MjY1ODQwIiwibWVzc2FnZXMiOlsiMHgwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDgwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDA2MjRkY2NiMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwYzAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwYTJlMDRmNWYwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwNjcwNzI2OTYzNjU3MzAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAzNDI1NDQzMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMCJdLCJzaWduYXR1cmVzIjpbIjB4ODM2MmE0NTY5OTcyODdhNmI4OWUyZGU1MmUyNmMyYWNhNDIzYWIwZWQ0MDFmOWEyM2M4MWRhMmUyYzU2YTVkYjI3MzY1YWRjYjQ3OGQ3YjM2NTU4ZGY1OGNhNWRkMjQwMTkxYTBmMDhhN2YwZWQ3OWVlMjNjZWM3NzUyMWU1YzIwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDFiIl0sInByaWNlcyI6eyJCVEMiOiI0MzcyMS43NSJ9fQo=

3. Submit the message to the chain​

When submitting the OracleDataSubmission, make sure to specify the source field as ORACLE_SOURCE_OPEN_ORACLE.

vegawallet transaction send \
    --wallet oracle-wallet \
    --pubkey 123abc \
    --network network-name \
    '{"oracleDataSubmission": { "source": "ORACLE_SOURCE_OPEN_ORACLE", "payload":"INSERT_PAYLOAD_STRING" }}'

You will be able to see this data by querying the API for OracleData. In the API response you will be able to check which markets had filters that matched this data.

Querying the data​

The Oracle Data list REST endpoint shows previous data submissions, which can be useful for confirming that data submission was sucessful, and/or determining the fields that a market's oracle spec requires.

JSON signed message data​

JSON ↗ messages are a simpler, more configurable alternative to Open Oracle data. They can be totally custom objects, as long as they are valid JSON. As they are not attested by any off-chain source in the way that Open Oracle messages are, and so it's generally advisable to check for an Open Oracle price source before choosing JSON data. The Vega key that signs the message will be referred to as the source for the data.

Using JSON signed message data in a market proposal​

For the binding, use the name field of the data. In the following example, the market is settled based on the number of people who have walked on the moon.

"oracleSpecBinding": {
  "settlementDataProperty": "moonwalkers",
  "tradingTerminationProperty": "vegaprotocol.builtin.timestamp"
}

The Oracle Specification that would bind to the moonwalkers property would be as follows:

   "oracleSpecForSettlementData": {
        "signers": [{ "pubKey":{ "key": "123abc" }}],
        "filters": [{
            "key": {
                "name": "moonwalkers",
                "type": "TYPE_INTEGER",
                "numberDecimalPlaces": "0"
            },
            "conditions": [{
                "operator": "OPERATOR_GREATER_THAN",
                "value": "12",
            }]
        }]
    }

Submitting JSON data​

Use the command line to submit a JSON message as a transaction that is signed by your Vega wallet and sent to the validators for consensus.

API note
  • Data should be encoded as strings. true should be "true", 12 should be "12"
  • In the API responses, the pubKeys field for JSON oracle data submissions is set to the VEGA public key of the submitter.

1. Define your JSON structure​

JSON data should be submitted as a single object of attributes and primitive values (i.e. no objects or arrays). Exactly what the attributes are called is up to the submitter of the data. Pick your structure in advance and ensure that it's well communicated. For this tutorial, we'll create a JSON data source for the number of humans that have walked on the moon:

{
  "moonwalkers": "12"
}

2. Encode the message​

All OracleDataSubmission data is base64 encoded. Here's how to do that on Linux or OSX:

echo '{"moonwalkers":"12"}' | base64

This will give you something like:

eyJtb29ud2Fsa2VycyI6IjEyIn0K

3. Submit the message to the chain​

When submitting the OracleDataSubmission, make sure to specify the source field as ORACLE_SOURCE_JSON.

Linux/OSX command line example
vegawallet transaction send \
    --wallet oracle-wallet \
    --pubkey 123abc \
    --network network-name \
    '{"oracleDataSubmission": { "source": "ORACLE_SOURCE_JSON", "payload":"RESPONSE_PAYLOAD" }}'

Querying an existing data source spec​

The Oracle Data list REST endpoint shows previous data submissions, which can be useful for confirming that a data submission was successful, and/or determining the fields that a market's data source spec requires.

Built-in data source​

Vega provides a timestamp source, which is useful for terminating a market at a set date. vegaprotocol.builtin.timestamp provides a Unix timestamp of the Vega time, which is to say the time agreed via consensus.

As the name implies, a built in data source is generated inside Vega, and cannot be submitted by other keys.

Using built-in data for trading termination​

It's possible to settle on any data source field - for instance checking if a boolean is true - but time is a good starting point, and the built-in time data source can be used for exactly that.

When using the built-in time source, use greater than or equals, rather than solely equals. This will help to avoid missing the time if no event is emitted with the precise required timestamp.

"oracleSpecForTradingTermination": {
    // pubKeys is empty as this is using a built in oracle
    "pubKeys": [],
    "filters": [{
        "key": {
            "name": "vegaprotocol.builtin.timestamp",
            "type": "TYPE_TIMESTAMP",
        },
        "conditions": [{
            "operator": "OPERATOR_GREATER_THAN_OR_EQUAL",
            "value": "1660826549",
        }]
    }]
}

This spec would make the market cease trading when the built-in time data source posted a Vega timestamp update that was on or after Thu Mar 31 2022 at 00:00:00.