Test your actors and tasks with multiple inputs, and expected outputs, integrating with results checker
By leveraging Jasmine, the extensible expect and Apify SDK, you can test tasks and actors,
and check for their output consistency and/or duplicates.
It goes well with monitoring suit for running your production runs, but this actor should be run in a scheduled manner for best results.
- You can run many tests in parallel or test them in series (as your account memory allows)
- You can run tests locally but accessing platform storage and actors
- Abstracts access to two other public actors:
The testing interface is familiar with Jasmine BDD tests, but with Apify specific async matchers:
({
it,
run,
expectAsync,
input, // Object containing the current input, you can access customData here
describe, // describe subsections
expect, // default Jasmine expect
_, // lodash as a helper to traverse array items and objects
moment // Moment.JS to help with dates and time math
}) => {
// describe is not needed, but it's good to keep everything tidy
describe('sub', () => {
it('should have preconfigured task working', async () => {
const myTaskResult = await run({
// actorId: 'actor/from-store', // can use an actorId directly
taskId: 'myuser/my-task-name',
input: {
some: 'extra input' // optional overrides
},
options: {
timeout: 15000 // optional call options
},
name: 'should have preconfigured task working'
});
// sync assertions, not very useful, expections should have inside async assertions
expect(myTaskResult.runId).not.toBeEmptyString();
/**
* Async assertions calls resources on the platform
*/
// reads the OUTPUT key
await expectAsync(myTaskResult).withOutput(async ({ contentType, value }) => {
expect(contentType)
// withContext give more information about of what you're testing
.withContext(myTaskResult.format('Body should be utf-8 JSON'))
.toEqual('application/json; charset=utf-8');
expect(value).toEqual({ hello: 'world' }, myTaskResult.format('Output body'));
});
// reads any key, fails the test if not found
await expectAsync(myTaskResult).withKeyValueStore(async ({ key, contentType, value }) => {
expect(value).toEqual({ status: true });
}, { keyName: 'INPUT' });
// gets requestQueue information
await expectAsync(myTaskResult).withRequestQueue(async ({
// contains everything from RequestQueueInfo
id, userId, createdAt,
modifiedAt, accessedAt, expireAt,
totalRequestCount, handledRequestCount, pendingRequestCount,
actId, actRunId, hadMultipleClients
}) => {
expect(totalRequestCount).toBeGreaterThan(1);
});
// check log for errors
await expectAsync(myTaskResult).withLog((log) => {
expect(log).not.toContain('ReferenceError');
expect(log).not.toContain('TypeError');
expect(log).not.toContain('The function passed to Apify.main() threw an exception');
});
// Check for dataset consistency
await expectAsync(myTaskResult).withChecker(({ runResult, output }) => {
expect(output.badItemCount).toBe(0);
}, {
functionalChecker: () => ({
myField: (field) => typeof field === 'string'
})
});
// Check for duplicate items
await expectAsync(myTaskResult).withDuplicates(({ runResult, output }) => {
expect(output).toEqual({});
}, {
taskId: 'myTaskId'
})
});
});
}Supports all extra Jasmine matchers, including asymmetrical matchers from https://github.com/JamieMason/Jasmine-Matchers
To access any without the JS editor complaining on the platform, you need to use global.any[asymmetricMatcher]
The special run parameter gives you the hability to run your tasks or actors, and return an accessor for their resources:
const result = await run({
taskId: 'xxx', // task either by id or using user/task-name
actorId: 'xxx', // actor either by id or using user/actor-name
input: {} // custom input override
options: {} // specific memory, timeout options
nonce: '1' // additional nonce for tasks running with the same input and options
name: 'run name'// give the run a name to be able to distinguish between them
});The run is idempotent and will run the same tasks once per test, but you can specify the nonce to force running it everytime
Those async matchers are lazy and only evaluated when you use them. You should use the result from run function to run expectAsync() on.
They abstract many common platform API calls. All callbacks can be plain closures or async ones, they are awaited anyway.
You also have full access to the Apify variable inside your tests.
Checks for the proper run status
Run expectations on the logContent
Ensures that no duplicates are found. You can provide a taskId with a pre-configured task or you can
provide all the input manually according to the docs here
By default, anything above 2 counted items are considered duplicates
Returns the OUTPUT of the run, containing an object like this:
{
// the keys here mean all the values that were found on the target dataset
"$$": {
"count": 4,
"originalIndexes": [
0,
12,
13,
15
],
"outputIndexes": [
9,
10,
11,
13
]
},
"MISSING!": { // this means it's missing or null value
"count": 8,
"originalIndexes": [
1,
3,
4,
6,
10,
14,
16,
17
],
"outputIndexes": [
0,
1,
2,
5,
8,
12,
14,
15
]
},
"$$$": {
"count": 4,
"originalIndexes": [
2,
5,
7,
8
],
"outputIndexes": [
3,
4,
6,
7
]
}
}withChecker((result: { runResult: Object, output: Object }) => void, input: Object, options?: Object)
Input is required and you need at least a taskId parameter pointing to a
pre-configured results-checker task or you can pass everything to the input.
Check the docs here
Options is the Apify.call/callTask options
Returns the OUTPUT of the run, containing an object like this:
"totalItemCount": 17,
"badItemCount": 0,
"identificationFields": [],
"badFields": {},
"extraFields": {},
"totalFieldCounts": {
"categories": 17,
"info": 17,
"likes": 17,
"messenger": 17,
"posts": 17,
"priceRange": 10,
"title": 17,
"pageUrl": 17,
"address": 17,
"awards": 17,
"email": 15,
"impressum": 17,
"instagram": 2,
"phone": 15,
"products": 17,
"transit": 4,
"twitter": 1,
"website": 16,
"youtube": 0,
"mission": 17,
"overview": 17,
"payment": 2,
"checkins": 12,
"#startedAt": 17,
"verified": 0,
"#url": 17,
"#ref": 17,
"reviews": 14,
"#version": 17,
"#finishedAt": 17
},
"badItems": "https://api.apify.com/v2/key-value-stores/_/records/BAD-ITEMS?disableRedirect=true"Returns dataset information and the items. Options can be optionally passed to limit the number of items returned,
using unwind parameter, or any other option that is available here: Dataset getItems
The dataset object contains:
{
items: [ [Object] ],
total: 1,
offset: 0,
count: 1,
limit: 999999999999
}The info object contains:
{
id: '',
userId: '',
createdAt: 2020-12-05T18:44:45.041Z,
modifiedAt: 2020-12-05T18:44:50.515Z,
accessedAt: 2020-12-05T18:44:50.515Z,
itemCount: 1,
cleanItemCount: 1,
actId: '',
actRunId: '',
stats: {
uploadedBytes: 0,
downloadedBytes: 0,
deflatedBytes: 0,
inflatedBytes: 21,
s3PutCount: 0,
s3GetCount: 0,
s3DeleteCount: 0,
readCount: 0,
writeCount: 1
}
}N.B.: this method waits at least 12 seconds to be able to read from the remote storage and make sure
it's ready to be accessed after the task/actor has finished running using run
Returns the OUTPUT key of the run. Can have any content type, check the contentType
Returns the SDK_CRAWLER_STATISTICS_0 key of the run by default, unless provided with another index
in the options.
Returns an object like this:
{
"requestsFinished": 217,
"requestsFailed": 99,
"requestsRetries": 0,
"requestsFailedPerMinute": 3,
"requestsFinishedPerMinute": 8,
"requestMinDurationMillis": 3071,
"requestMaxDurationMillis": 41800,
"requestTotalFailedDurationMillis": 686856,
"requestTotalFinishedDurationMillis": 3161769,
"crawlerStartedAt": "2020-12-07T05:06:44.107Z",
"crawlerFinishedAt": null,
"statsPersistedAt": "2020-12-07T05:34:04.209Z",
"crawlerRuntimeMillis": 1640402,
"crawlerLastStartTimestamp": 1607317603807,
"requestRetryHistogram": [
316
],
"statsId": 0,
"requestAvgFailedDurationMillis": 6938,
"requestAvgFinishedDurationMillis": 14570,
"requestTotalDurationMillis": 3848625,
"requestsTotal": 316
}withKeyValueStore((output: { value: any, contentType: string }) => void, options: { keyName: string })
Returns the content of the selected keyName. The test fails if the key doesn't exist.
You can access the INPUT that was used for the run using { keyName: 'INPUT' }
Access the requestQueue object, that contains:
{
id: '',
userId: '',
createdAt: 2020-12-05T18:44:45.048Z,
modifiedAt: 2020-12-05T18:44:45.048Z,
accessedAt: 2020-12-05T18:44:45.048Z,
expireAt: 2021-02-03T18:44:45.048Z,
totalRequestCount: 0,
handledRequestCount: 0,
pendingRequestCount: 0,
actId: '',
actRunId: '',
hadMultipleClients: false
}N.B.: all those exists only on expectAsync and need to be awaited, as demonstrated above:
await expectAsync(runResult).withDataset((something) => {
expect(something).toEqual('here');
});jasmine.any() and jasmine.anything() can be accessed using global.jasmine
The tests output are available in the key value store under OUTPUT key, with the following structure:
{
"suite2": {
"id": "suite2",
"description": "one",
"fullName": "Actor tests one",
"failedExpectations": [],
"deprecationWarnings": [],
"duration": 26484,
"properties": null,
"status": "passed",
"specs": [
{
"id": "spec0",
"description": "should work",
"fullName": "Actor tests one should work",
"failedExpectations": [],
"passedExpectations": [
{
"matcherName": "toHaveStatus",
"message": "Passed.",
"stack": "",
"passed": true
},
{
"matcherName": "toEqual",
"message": "Passed.",
"stack": "",
"passed": true
},
{
"matcherName": "withDataset",
"message": "Passed.",
"stack": "",
"passed": true
},
{
"matcherName": "withRequestQueue",
"message": "Passed.",
"stack": "",
"passed": true
},
{
"matcherName": "withOutput",
"message": "Passed.",
"stack": "",
"passed": true
},
{
"matcherName": "withKeyValueStore",
"message": "Passed.",
"stack": "",
"passed": true
},
{
"matcherName": "withChecker",
"message": "Passed.",
"stack": "",
"passed": true
}
],
"deprecationWarnings": [],
"pendingReason": "",
"duration": 26480,
"properties": null,
"status": "passed"
}
]
},
"suite3": {
"id": "suite3",
"description": "two",
"fullName": "Actor tests two",
"failedExpectations": [],
"deprecationWarnings": [],
"duration": 21,
"properties": null,
"status": "passed",
"specs": [
{
"id": "spec1",
"description": "works",
"fullName": "Actor tests two works",
"failedExpectations": [
{
"matcherName": "toBe",
"message": "Expected true to be false.",
"stack": "Error: Expected true to be false.\n at <Jasmine>\n at listOnTimeout (internal/timers.js:549:17)\n at processTimers (internal/timers.js:492:7)",
"passed": false,
"expected": false,
"actual": true
}
],
"passedExpectations": [],
"deprecationWarnings": [],
"pendingReason": "",
"duration": 15,
"properties": null,
"status": "failed"
}
]
}
}This is a very lightweight actor that only intermediates actor runs, it can be run with the lowest amount of memory, which is 128MB. Running for an hour should consume around 0.125 CUs.
Automated and integration tests are a must have for any complex piece of software. For Apify actors, it's no different. Apify actors can be one (or many inputs) to one output, or it can have many items (through the dataset).
Apache 2.0