Archive Paging: Difference between revisions

(21 intermediate revisions by 3 users not shown)

Line 1:

== Overview ==

~~To speed up~~ the ~~time it takes~~ to ~~refresh data, OdataLink's models include an archive. You can~~ control how ~~the archive works by changing the '''Archive Paging''' option~~.

OdataLink gives you the ability to control how your data is requested from Xero or MYOB.

The Paging By setting of the model allows you to control how the data is paged or sliced into smaller requests.

[[File:model_performance_options.png|thumb|right|400px]]

These smaller requests get chained one after the other and the complete set of data will still be returned.

However, because each request handles smaller amounts of data, it can give you performance gain or allow you to download larger sets of data.

The '''Paging By''' setting allows you to request data from MYOB or Xero, paged '''By Month, By Year or by Default'''.

''Please note that '''Paging By''' setting may download your data over a '''larger number of calls'''''.

''If you are receiving Timeout errors or Xero daily call limit reached [[HelpID:ODataFeedStatus9004|Error 9004]], please create alternative model architecture to include '''Static Archive''' type to cache larger Historical dataset, or configure Archive by Cut Off Date in [[Archive Types]].''

== How to Configure the Paging By setting for an OdataLink model ==

Archive Paging setting can be found within "Performance Options" in Model Settings.

# Navigate to your OdataLink dashboard and click on '''[[Models]]''' on the left side panel.

# Click on '''View''' button corresponding to the model that you need to configure.

# Under '''Settings''' Tab, click on '''Edit Model''' button.

# Configure '''Archive Paging''' setting under '''Performance Options'''.

For more detailed instructions on how to configure your model, please refer to [[Editing a model's details | Model Configurations]].

== Understanding the Paging By setting ==

The possible values are:

* Default

Line 11:

Line 36:

=== Default ===

When the '''~~Archive~~ Paging''' is set to '''Default ''', OdataLink will request the data as instructed. This is the legacy behaviour of OdataLink. The filters specified will be sent as-is to your data file.

When the '''Paging''' is set to '''Default ''', OdataLink will request the data as instructed. This is the legacy behaviour of OdataLink. The filters specified will be sent as-is to your data file.

While most customers will benefit from using the 'By Month' or 'By Year' option, we kept this legacy mechanism for very small data files or for those that use complex queries.

While most customers will benefit from using the '''By Month''' or '''By Year''' option, we kept this legacy mechanism for very small data files or for those that use complex queries.

=== By Month ===

When the '''~~Archive~~ Paging''' is set to '''By Month''', OdataLink to query the data one month at a time.

When the '''Paging''' is set to '''By Month''', OdataLink to query the data one month at a time.

=== By Year ===

When the '''~~Archive~~ Paging''' is set to '''By Year''', OdataLink to query the data one year at a time.

When the '''Paging''' is set to '''By Year''', OdataLink to query the data one year at a time.

== Recommendations ==

Line 29:

Line 55:

=== Recommendations for MYOB ===

MYOB's API is more designed to handle small volume of data. Because of this, using the 'By Year' or 'By Month' option can provide a lot of performance gains.

MYOB's API is more designed to handle small volume of data. Because of this, using the '''By Year''' or '''By Month''' option can provide a lot of performance gains.

We recommend you use the 'By Year' if you have 2,000 transactions per year and require 3 or more years of data.

We recommend you use the '''By Year''' if you have 2,000 transactions per year and require 3 or more years of data.

We recommend you use the 'By Month' if you have 1,000 transactions per month and require multiple months of data.

We recommend you use the '''By Month''' if you have 1,000 transactions per month and require multiple months of data.

In our tests, pulling 20,000 transactions across 10 years from MYOB took:

Line 41:

Line 67:

We strongly recommend you test the different option and choose the one that provides the best performance for your MYOB data file.

See [[Editing a model's details]] for detailed instructions on how to configure your model.

=== Recommendations for Xero ===

[[https://developer.xero.com/documentation/api/accounting/overview Xero's API]] is very well structured to handle large volumes of data.

[https://developer.xero.com/documentation/api/accounting/overview Xero's API] is very well structured to handle large volumes of data.

However, Xero have introduced limits to the number of transactions that can be requested at a single time. If you need to report on more than 100,000 transactions, then using the 'By Month' or 'By Year' option is recommended.

However, Xero have introduced limits to the number of transactions that can be requested at a single time. If you need to report on more than 100,000 transactions, then using the '''By Month''' or '''By Year''' option is recommended.

These options will ensure that each request stay below the 100,000 transactions cap.

~~You can find further information about the record limits that Xero imposes via the following page~~.

See [[Editing a model's details]] for detailed instructions on how to configure your model.

[[https://developer.xero.com/documentation/api/efficient-data-retrieval#high-volume-threshold-limits ~~https://developer.xero.com/documentation/api/efficient-data-retrieval#high-volume-~~threshold-limits]]

See [https://developer.xero.com/documentation/api/efficient-data-retrieval#high-volume-threshold-limits Xero High Volume threshold limits] on the Xero API site for further information.

@@ Line 1: / Line 1: @@
 == Overview ==
-To speed up the time it takes to refresh data, OdataLink's models include an archive.  You can control how the archive works by changing the '''Archive Paging''' option.
+OdataLink gives you the ability to control how your data is requested from Xero or MYOB.
+The Paging By setting of the model allows you to control how the data is paged or sliced into smaller requests.
+[[File:model_performance_options.png|thumb|right|400px]]
+These smaller requests get chained one after the other and the complete set of data will still be returned.
+However, because each request handles smaller amounts of data, it can give you performance gain or allow you to download larger sets of data.
+The '''Paging By''' setting allows you to request data from MYOB or Xero, paged '''By Month, By Year or by Default'''.
+''Please note that '''Paging By''' setting may download your data over a '''larger number of calls'''''.
+''If you are receiving Timeout errors or Xero daily call limit reached [[HelpID:ODataFeedStatus9004|Error 9004]], please create alternative model architecture to include '''Static Archive''' type to cache larger Historical dataset, or configure Archive by Cut Off Date in [[Archive Types]].''
+== How to Configure the Paging By setting for an OdataLink model ==
+Archive Paging setting can be found within "Performance Options" in Model Settings.
+# Navigate to your OdataLink dashboard and click on '''[[Models]]''' on the left side panel.
+# Click on '''View''' button corresponding to the model that you need to configure.
+# Under '''Settings''' Tab, click on '''Edit Model''' button.
+# Configure '''Archive Paging''' setting under '''Performance Options'''.
+For more detailed instructions on how to configure your model, please refer to [[Editing a model's details | Model Configurations]].
+== Understanding the Paging By setting ==
 The possible values are:
 * Default
@@ Line 11: / Line 36: @@
 === Default ===
-When the '''Archive Paging''' is set to '''Default ''', OdataLink will request the data as instructed. This is the legacy behaviour of OdataLink. The filters specified will be sent as-is to your data file.
+When the '''Paging''' is set to '''Default ''', OdataLink will request the data as instructed. This is the legacy behaviour of OdataLink. The filters specified will be sent as-is to your data file.
-While most customers will benefit from using the 'By Month' or 'By Year' option, we kept this legacy mechanism for very small data files or for those that use complex queries.
+While most customers will benefit from using the '''By Month''' or '''By Year''' option, we kept this legacy mechanism for very small data files or for those that use complex queries.
 === By Month ===
-When the '''Archive Paging''' is set to '''By Month''', OdataLink to query the data one month at a time.
+When the '''Paging''' is set to '''By Month''', OdataLink to query the data one month at a time.
 === By Year ===
-When the '''Archive Paging''' is set to '''By Year''', OdataLink to query the data one year at a time.
+When the '''Paging''' is set to '''By Year''', OdataLink to query the data one year at a time.
 == Recommendations ==
@@ Line 29: / Line 55: @@
 === Recommendations for MYOB ===
-MYOB's API is more designed to handle small volume of data. Because of this, using the 'By Year' or 'By Month' option can provide a lot of performance gains.
+MYOB's API is more designed to handle small volume of data. Because of this, using the '''By Year''' or '''By Month''' option can provide a lot of performance gains.
-We recommend you use the 'By Year' if you have 2,000 transactions per year and require 3 or more years of data.
+We recommend you use the '''By Year''' if you have 2,000 transactions per year and require 3 or more years of data.
-We recommend you use the 'By Month' if you have 1,000 transactions per month and require multiple months of data.
+We recommend you use the '''By Month''' if you have 1,000 transactions per month and require multiple months of data.
 In our tests, pulling 20,000 transactions across 10 years from MYOB took:
@@ Line 41: / Line 67: @@
 We strongly recommend you test the different option and choose the one that provides the best performance for your MYOB data file.
+See [[Editing a model's details]] for detailed instructions on how to configure your model.
 === Recommendations for Xero ===
-[[https://developer.xero.com/documentation/api/accounting/overview Xero's API]] is very well structured to handle large volumes of data.
+[https://developer.xero.com/documentation/api/accounting/overview Xero's API] is very well structured to handle large volumes of data.
-However, Xero have introduced limits to the number of transactions that can be requested at a single time. If you need to report on more than 100,000 transactions, then using the 'By Month' or 'By Year' option is recommended.
+However, Xero have introduced limits to the number of transactions that can be requested at a single time. If you need to report on more than 100,000 transactions, then using the '''By Month''' or '''By Year''' option is recommended.
 These options will ensure that each request stay below the 100,000 transactions cap.
-You can find further information about the record limits that Xero imposes via the following page.
+See [[Editing a model's details]] for detailed instructions on how to configure your model.
-[[https://developer.xero.com/documentation/api/efficient-data-retrieval#high-volume-threshold-limits https://developer.xero.com/documentation/api/efficient-data-retrieval#high-volume-threshold-limits]]
+See [https://developer.xero.com/documentation/api/efficient-data-retrieval#high-volume-threshold-limits Xero High Volume threshold limits] on the Xero API site for further information.