CollectApi

CollectApi

  • Docs
  • Template

›Template

Template

  • Introduction
  • Helpers
  • Usage Examples

Template Helpers

Helpers are there to make things easier or add some features that you can do with basic features.

Priority Order

Priority of helpers, and their sub keywords.

OrderSub Keys
__html
__request__next
__each__next, __limit, __filter
__if__then, __else
__num
__js__next
__json__next

We're adding more helpers and features according to the customers needs. If you think a helper would improve or help customers to create their templates easly you can contact us through [email protected]. Your suggestion must not break old shemas.

__if helper

__if helper uses two more paramaters. They are __then and __else.

__if helper takes a string value that filled with javascript condition code like true, item.finished === 'yes', item.id > 1, item.name && item.name.lenght > 20, etc.

As you can see you can use javascript inside this string. If this value is true it'll use __then's value as generator, if not true it'll use __else's value as generator. If it looks for __then's or __else's value and can't find one it'll generate null.

For example; Let's create a template that uses https://jsonplaceholder.typicode.com/todos api. We want to print messages according to the todo's completed status. If it's completed we want to add Finished tag to start of items text data. In example we created there are two schemas. First one returns only the ones that has completed:true. Second one prints every one but adds Finished Work or Not Finished Work to start of the text according to the completed value.

Example with JsonPlaceholder Api

  1. We need to make a request with __request helper.
  2. We need to use __each to iterate the result of __request
  3. We check if item.completed to decide what to return.
  • Value of item(cursor) inside __each operation would be iterated item in array.
  • If you want to access to scope.body object when cursor is changed. You can use scope.body object directly. For example __if: "scope.body.name === 'test'".
  • __each and __if helpers are in the same object. In this scenario template would follow the order of Priority Order. Work flow would be __each -> __if
module.exports = {
    __request: {
        uri: 'https://jsonplaceholder.typicode.com/todos',
        then: {
            __each: '',
            __limit: 10,
            __if: 'item.completed',
            __then: 'Completed / #{{id}} {{title}}',
            __else: 'Ongoing   /#{{id}} {{title}}',
        },
    },
};
  • Response
[
    "Ongoing   /#1 delectus aut autem",
    "Ongoing   /#2 quis ut nam facilis et officia qui",
    "Ongoing   /#3 fugiat veniam minus",
    "Completed / #4 et porro tempora",
    "Ongoing   /#5 laboriosam mollitia et enim quasi adipisci quia provident illum",
    "Ongoing   /#6 qui ullam ratione quibusdam voluptatem quia omnis",
    "Ongoing   /#7 illo expedita consequatur quia in",
    "Completed / #8 quo adipisci enim quam ut ab",
    "Ongoing   /#9 molestiae perspiciatis ipsa",
    "Completed / #10 illo est ratione doloremque quia maiores aut"
]

Example with Query Paramaters

  • Value of item(cursor) here would be scope.body object as there isn't any helper to change cursor position.
module.exports = {
    __if: "item.name === 'test'",
    __then: {
        success: true,
        message: 'Welcome {{name}}',
    },
    __else: {
        success: false,
        message: 'Wrong name {{name}}!',
    },
};
  • Response of https://api.collectapi.com/run/test/5c6e49797c85557548c8dc8b?name=test
{
    "success": true,
    "user": "test"
}
  • Response of https://api.collectapi.com/run/test/5c6e49797c85557548c8dc8b?name=fake
{
    "success": false,
    "user": "fake"
}

__each helper

This is a foreach operation. It's value should point to a json array. If your current cursor is an array __each's value should be "".

__limit with __each

This is a helper to put a limit to how much item would you like to return maximum. Value must be a integer.

__filter with __each

This is a helper to filter arrays. It works same as __if helper but would be applied to each element before processing __each helper.

List Post Example

In this example we would take title and an object list of {x: <number>}.

We would like to reformat our data and return only odd numbers and return limit the result array to max 2 length.

To achive this we need to use __filter helper to apply similar logic to __if but here it would be applied to iterated array item.

module.exports = {
    title: '{{title}}',
    result: {
        __each: 'list',
        __filter: 'item.x % 2',
        __limit: 2,
        __next: { __num: '{{x}}' },
    },
};
  • Request
curl -X POST \
  https://api.collectapi.com/run/test/5c6e63cad6f15f5422edd760 \
  -H 'Content-Type: application/json' \
  -d '{
  "title":"collectapi template",
  "list":[{"x":1},{"x":2},{"x":3},{"x":4},{"x":5},{"x":6},{"x":7},{"x":8}]
}'
  • Response
{
    "title": "collectapi template",
    "result": [1, 3]
}

Using scope.body Example

In this example we would be multiply the items in the array by the value of body.times.

  • {{$<javascript-code>}} is used to run javascript code inside string parser.
  • Inside __each operation cursor would be iterated array item so to access times value we used scope.body.times.
  • We can use __num or __json to return our desired value. __num can only return numbers. __json can return any value type that is supported by json.
module.exports = {
    message: 'f(x) = x * {{times}}',
    result: {
        __each: 'list',
        __json: '{{$item.x * scope.body.times}}',
    },
};
  • Request
curl -X POST \
  https://api.collectapi.com/run/test/5c6e66e840bb6353e1b9c295 \
  -H 'Content-Type: application/json' \
  -d '{
  "times":5,
  "list":[{"x":1},{"x":2},{"x":3}]
}'
  • Response
{
    "message": "f(x) = x * 5",
    "result": [5, 10, 15]
}

__next with __each

This is a helper to wrap what should be used for __each iteration elements.

__request helper

This is a tool to call external services. You can find an example of it at right section.

Some more info about it;

KeyDescriptionDefaultTypeRequired
uriRequest UrlNo DefaultStringYes
methodWhich method used to call your uri call. GET or POSTGETStringNo
bodyBody data if we want to post something.{}ObjectNo
thenUses the response from request to continue with template.No DefaultSchemaNo
headersHeaders used with request.{}ObjectNo

Some tips and details:

  1. If you don't provide any then object __request would return the response directly.

Get Example

module.exports = {
    __request: {
        uri: 'https://jsonplaceholder.typicode.com/todos',
        then: {
            __each: '',
            __limit: 3,
            text: '{{title}}',
        },
    },
};
  • Response
[
  {
    "text": "delectus aut autem"
  },
  {
    "text": "quis ut nam facilis et officia qui"
  },
  {
    "text": "fugiat veniam minus"
  }
]

Post Example

module.exports = {
    __request: {
        uri: 'https://jsonplaceholder.typicode.com/todos',
        method: 'post',
        body: {
            title: 'product title {{name}}',
        },
    },
};
  • Response for https://api.collectapi.com/run/test/5c6e875c40bb6353e1b9c298?name=collectapi
{
    "id": 201,
    "title": "product title collectapi"
}

__next with __request

You can also use __next with __request. __next is helpful to break Priority Order of helpers.

For example if you write a template like this to parse html it wouldn't work. Because the priority of __html is higher than __request. It would try to parse html before getting the html from request.

// Would't work correctly
module.exports = {
    __request: {
        uri: 'https://www.lipsum.com/feed/html',
    },
    __html: {
        title: '{{#Inner > h1}}',
    },
    result: '{{title}}',
};
{
    "result": ""
}

To fix this problem we would move rest of the request to inside of __next keyword. Like this we are telling the template that we want to first make the request then parse it.

module.exports = {
    __request: {
        uri: 'https://www.lipsum.com/feed/html',
    },
    __next: {
        __html: {
            title: '{{#Inner > h1}}',
        },
        result: '{{title}}',
    },
};
{
    "result": "Lorem Ipsum"
}

You could also use then keyword of __request to parse html like this.

__num helper

This is a helper to parse numbers from string. In example we put static "0.2" string but you can put the number with "{{number}}".

  • Example
module.exports = {
    parsed_number: { __num: '0.2' },
};
  • Response
{
    "parsed_number": 0.2
}

__json helper

This is a helper to parse json from string.

  • Example
module.exports = {
    parsed_json: { __json: '{"test":"test text"}' },
};
  • Response
{
    "parsed_number": { "test": "test text" }
}

__next with __json

This is a helper to continue schema with data parsed from json string. In example we put static "{\"test\":\"test text\"}" string but you can define it with "{{json}}" with external stringfied text.

  • Code
module.exports = {
    parsed_json: {
        __json: '{"test":"test text"}',
        __next: {
            next_next: 'last {{test}}',
        },
    },
};
  • Response
{
    "parsed_number": { "next_next": "last test text" }
}

__js helper

Sometimes you would like to run javascript code and return it's value as it is.

  1. To do this you can do something like this key: {__json:'{{$ 15+22 }}'}.
  2. Or to shorten it you can do key: {__js:'15+22'}. __js works just like __json but without the need of boilerplate {{$<code>}}.
  3. You can even shorten it and write key: __js('15+22').
  4. __js function accepts a second paramater to pass as __next value. So key: __js('15+22','Result: {{this}}') would be equal to key: {__js: '15+22',__next: 'Result: {{this}}'}. Inside __next cursor would be result of result of __js response.
  • Example
function test() {
    return new Date();
}

module.exports = {
    result: __js('15+22'),
    date: __js('test()'),
};
  • Response of https://api.collectapi.com/run/test/5c6ff84327f3e6414008d1ff
{
    "result": 37,
    "date": "2019-02-22T13:25:36.933Z"
}
  • Example with __next.
module.exports = {
    with_function: __js('15+22', 'Result: {{this}}'),
    with_object: { __js: '15+22', __next: 'Result: {{this}}' },
};
  • Response of https://api.collectapi.com/run/test/5c6ff91c7b1f41410b5698e8
{
    "with_function": "Result: 37",
    "with_object": "Result: 37"
}

__html helper

This is a parser with different rules. Basic "{{key}}" rule still works. But instead of key's we need to give it some css selector or javascript code to get what we want. __html helpers would work before other helpers and replace current cursor with crawled data. The helpers work after \_\_html helper would use the cursor from it.

  • __html uses cheerio to parse html. You check their documentation for more information.
  • You can use either double braces to read data {{}}. But here cursor can work with cheerio to read text or run custom javascript.
  • {{#Inner > h1}} would try to find #Inner > h1 in document and return it's text value. It'd return $('#Inner > h1').text().
  • If you want to modify process and return something like attribute you can use $ to access javascript side. {{#Inner > h1}} also can be {{$ $('#Inner > h1').text() }}. First $ inside of double braces for running javascript code, second $ is cheerio root object.
  • For example you can access to src attribute of img dom object like {{$$('img').attr('src')}}
  • __each inside __html also works differently for more information check __each with __html section.

Html Parse Example

  • In this example we're using __each to get an array of documents then uses them to define values for paragraphs array.
  • In example there is 2 different usage way to get data first is {{#Inner > h1}} and second is {{$$(item).text()}}. Actualy first one is almost same as second but it's short way to write and don't need much knowledge about cheerio. {{#Inner > h1}} would be equal to {{$$('#Inner > h1').text()}} if it's not inside in an __each helper. If it's inside of a __each helper it would mean {{$$(item).find('#Inner > h1').text()}}.
module.exports = {
    __request: {
        uri: 'https://www.lipsum.com/feed/html',
        then: {
            __html: {
                title: '{{#Inner > h1}}',
                paragraphs: {
                    __each: '#lipsum p',
                    text: '{{$$(item).text()}}',
                },
            },
        },
    },
    source: 'from {{title}}',
    results: {
        __each: 'paragraphs',
        __next: '{{text}}',
    },
};
  • Response
{
    "source": "from Lorem Ipsum",
    "results": [
        "Lorem ipsum dolor sit amet, consectetur...",
        "Vestibulum sed arcu nec odio condimentum ...",
        "Nam porta est sapien, ut suscipit est so...",
        "Integer consectetur pretium ligula ac convallis..."
    ]
}

__each with __html

  • Above example we used __each with __html to iterate over #lipsum p document objects.
  • __each would accept cheerio supported css selectors to iterate over.
  • Cursor inside __each loop would be try to find the selector under the iterated item like {{query}} would act like $(item).find(query).text().
  • If you want to use javascript under __each loop item would give you iterated item. For example you can use {{$ $(item).find('img').attr('src') }} something like this code to find image url under iterated list.

Note: Inside __each you can't use {{}}. You need to give it css selector.

How to use advanced features

You can use any features from popular nodejs library for html dom parsing cheerio (Go to Cheerio docs). To use it you need to put a $ sembol to start of {{}} paramater system like {{$}} then you can write cheerio code inside of it. You can access to current cursor object with item variable like {{$$(item).find('img').attr('src')}} this would search for img tagged element and get it's src attribute. For more information you can check cheerio docs.

Accessible Libraries

Libraries that can be accessed from inside of javascript code are as follow

  • Moment.js (moment) docs
  • Moment-Business.js (business) docs
  • lodash (_) docs
  • he (he) docs

Also __html uses cheerio to parse html. You check their documentation for more information.

← IntroductionUsage Examples →
  • Priority Order
  • __if helper
    • Example with JsonPlaceholder Api
    • Example with Query Paramaters
  • __each helper
    • __limit with __each
    • __filter with __each
    • List Post Example
    • Using scope.body Example
    • __next with __each
  • __request helper
    • Get Example
    • Post Example
    • __next with __request
  • __num helper
  • __json helper
    • __next with __json
  • __js helper
  • __html helper
    • Html Parse Example
    • __each with __html
    • How to use advanced features
  • Accessible Libraries
CollectApi
Docs
CollectApi DocumentationTemplate Documentation
More
CollectApi
Copyright © 2024 CollectApi