Archive Paging
Overview
OdataLink gives you the ability to control how your data is requested from Xero or MYOB.
The Paging By setting of the model allows you to control how the data is paged or sliced into smaller requests.
These smaller requests get chained one after the other and the complete set of data will still be returned.
However, because each request handles smaller amounts of data, it can give you performance gain or allow you to download larger sets of data.
The Paging By setting allows you to request data from MYOB or Xero, paged By Month, By Year or by Default.
Please note that Paging By setting downloads your data over a larger number of calls.
If you are receiving Timeout errors or Xero daily call limit reached Error 9004, please create alternative model architecture to include Static Archive type to cache larger Historical dataset.
How to Configure the Paging By setting for an OdataLink model
Archive Paging setting can be found within "Performance Options" in Model Settings.
- Navigate to your OdataLink dashboard and click on Models on the left side panel.
- Click on View button corresponding to the model that you need to configure.
- Under Settings Tab, click on Edit Model button.
- Configure Archive Paging setting under Performance Options.
For more detailed instructions on how to configure your model, please refer to Model Configurations.
Understanding the Paging By setting
The possible values are:
- Default
- By Month
- By Year
Default
When the Paging is set to Default , OdataLink will request the data as instructed. This is the legacy behaviour of OdataLink. The filters specified will be sent as-is to your data file.
While most customers will benefit from using the By Month or By Year option, we kept this legacy mechanism for very small data files or for those that use complex queries.
By Month
When the Paging is set to By Month, OdataLink to query the data one month at a time.
By Year
When the Paging is set to By Year, OdataLink to query the data one year at a time.
Recommendations
Recommendations for MYOB
MYOB's API is more designed to handle small volume of data. Because of this, using the By Year or By Month option can provide a lot of performance gains.
We recommend you use the By Year if you have 2,000 transactions per year and require 3 or more years of data.
We recommend you use the By Month if you have 1,000 transactions per month and require multiple months of data.
In our tests, pulling 20,000 transactions across 10 years from MYOB took:
- 11 minutes with the legacy 'Default' option
- 4.5 minutes with the 'By Month' option
- 3.5 minutes with the 'By Year' option.
We strongly recommend you test the different option and choose the one that provides the best performance for your MYOB data file.
See Editing a model's details for detailed instructions on how to configure your model.
Recommendations for Xero
Xero's API is very well structured to handle large volumes of data.
However, Xero have introduced limits to the number of transactions that can be requested at a single time. If you need to report on more than 100,000 transactions, then using the By Month or By Year option is recommended.
These options will ensure that each request stay below the 100,000 transactions cap.
See Editing a model's details for detailed instructions on how to configure your model.
See Xero High Volume threshold limits on the Xero API site for further information.