Reducing Data Usage and Optimising your OdataLink Feeds: Difference between revisions
Jump to navigation
Jump to search
mNo edit summary |
m fair use policy published, correcting the location of the policy doc on the website |
||
| (8 intermediate revisions by the same user not shown) | |||
| Line 7: | Line 7: | ||
* Improve refresh performance | * Improve refresh performance | ||
You can monitor your current usage at any time in your | You can monitor your current usage at any time in your '''OdataLink Admin Portal → Usage Stats'''. | ||
See [[How to track my OdataLink Usage]] for usage broken down by [[Data Files|Data file]], [[Models|Model]], and [[Users|User]]. | See [[How to track my OdataLink Usage]] for usage broken down by [[Data Files|Data file]], [[Models|Model]], and [[Users|User]]. | ||
| Line 14: | Line 14: | ||
== What is Usage == | == What is Usage == | ||
<onlyinclude> | |||
'''Usage''' is defined as the volume of data OdataLink retrieves from your connected [[Data Files|data files]] each month. | '''Usage''' is defined as the volume of data OdataLink retrieves from your connected [[Data Files|data files]] each month. | ||
This includes data retrieved by [[Microsoft Power BI|Power BI]], [[Microsoft Excel|Excel]], or [[OData Client Compatibility|other]] data pipeline and reporting tools, either via automatic scheduled refreshes or manual report refresh. | This includes data retrieved by [[Microsoft Power BI|Power BI]], [[Microsoft Excel|Excel]], or [[OData Client Compatibility|other]] data pipeline and reporting tools, either via automatic scheduled refreshes or manual report refresh. | ||
</onlyinclude> | |||
= Pick your Optimisation Layer | = Pick your Optimisation Layer = | ||
Optimisation techniques are grouped by the tooling you use. Pick the page that matches your role — or read all three if your organisation spans several layers. | Optimisation techniques are grouped by the tooling you use. Pick the page that matches your role — or read all three if your organisation spans several layers. | ||
| Line 28: | Line 29: | ||
| [[How to Reduce my OdataLink Usage - For Advanced Users]] || Power BI, Fabric, and Excel report developers (Power Query / M) || Query folding, OData <code>$filter</code> operators, <code>$select</code> projection, incremental refresh | | [[How to Reduce my OdataLink Usage - For Advanced Users]] || Power BI, Fabric, and Excel report developers (Power Query / M) || Query folding, OData <code>$filter</code> operators, <code>$select</code> projection, incremental refresh | ||
|- | |- | ||
| [[How to Reduce my OdataLink Usage - For Power Users]] || Data engineering teams (SSIS, ADF, Fabric Data Pipelines, Fabric Dataflow Gen2) || Stage-once-serve-many, watermarked incremental loads, historical backfill strategy, scheduling discipline, end-to-end cost monitoring | | [[How to Reduce my OdataLink Usage - For Power Users]] || Data engineering teams (SSIS, ADF, Fabric Data Pipelines, Fabric Dataflow Legacy/Gen2) || Stage-once-serve-many, watermarked incremental loads, historical backfill strategy, scheduling discipline, end-to-end cost monitoring | ||
|} | |} | ||
<span style="color:navy"><blockquote>'''Tip:''' Start with the [[How to Reduce my OdataLink Usage - For Everyday Users|Everyday Users]] page regardless of your role. The model-level settings described there apply to '''every''' refresh | <span style="color:navy"><blockquote>'''Tip:''' Start with the [[How to Reduce my OdataLink Usage - For Everyday Users|Everyday Users]] page regardless of your role. The model-level settings described there apply to '''every''' refresh, automatic or manual, report or pipeline, and deliver the largest reduction with the least amount of changes.</blockquote></span> | ||
<br /> | <br /> | ||
<!-- | |||
== For everyone (accountants, bookkeepers, business users) == | == For everyone (accountants, bookkeepers, business users) == | ||
| Line 57: | Line 58: | ||
<br> | <br> | ||
--> | |||
== Quick checklist == | == Quick checklist == | ||
A cross-cutting checklist drawn from all three pages. Detailed steps and tool-specific guidance are in the linked pages above. | A cross-cutting checklist drawn from all three pages. Detailed steps and tool-specific guidance are in the linked pages above. | ||
| Line 69: | Line 70: | ||
* [ ] Polling enabled | * [ ] Polling enabled | ||
''' | '''Advanced Users:''' | ||
* [ ] Date filters applied at the OData navigation step (not post-download) | * [ ] Date filters applied at the OData navigation step (not post-download) | ||
* [ ] Query folding verified via '''View Native Query''' | * [ ] Query folding verified via '''View Native Query''' | ||
| Line 75: | Line 76: | ||
* [ ] Incremental refresh enabled for time-series datasets | * [ ] Incremental refresh enabled for time-series datasets | ||
'''Data | '''Azure/Fabric/Data Pipeline developers:''' | ||
* [ ] OdataLink data staged once into SQL / Lakehouse / Warehouse / Dataflow Gen2; downstream consumers read from the staging layer | * [ ] OdataLink data staged once into SQL / Lakehouse / Warehouse / Dataflow Gen2; downstream consumers read from the staging layer | ||
* [ ] All recurring loads use <code>$filter</code> with a high-watermark column | * [ ] All recurring loads use <code>$filter</code> with a high-watermark column | ||
| Line 94: | Line 95: | ||
== Related Topics == | == Related Topics == | ||
* [https://odatalink.com/fair-use-policy OdataLink Fair Use Policy | * [https://odatalink.com/eula/fair-use-policy OdataLink Fair Use Policy] | ||
* [[How to track my OdataLink Usage]] | * [[How to track my OdataLink Usage]] | ||
* [[OData Feed URLs]] | * [[OData Feed URLs]] | ||
<!-- * [https://developer.xero.com/pricing Xero Developer Pricing] | <!-- * [https://developer.xero.com/pricing Xero Developer Pricing] | ||
* [https://developer.xero.com/faq/pricing-and-policy-updates Xero Pricing and Policy Updates FAQ]--> | * [https://developer.xero.com/faq/pricing-and-policy-updates Xero Pricing and Policy Updates FAQ]--> | ||
Latest revision as of 23:06, 4 June 2026
Overview
To keep OdataLink subscriptions affordable and predictable for everyone, we've introduced a Fair Use Policy. This policy sets a monthly download limit for data retrieved from your connected data file(s).
The guidance below applies to all models connecting to Xero, Xero Practice Manager, MYOB and QuickBooks. Following these steps will:
- Reduce your monthly download volume; and
- Improve refresh performance
You can monitor your current usage at any time in your OdataLink Admin Portal → Usage Stats.
See How to track my OdataLink Usage for usage broken down by Data file, Model, and User.
What is Usage
Usage is defined as the volume of data OdataLink retrieves from your connected data files each month.
This includes data retrieved by Power BI, Excel, or other data pipeline and reporting tools, either via automatic scheduled refreshes or manual report refresh.
Pick your Optimisation Layer
Optimisation techniques are grouped by the tooling you use. Pick the page that matches your role — or read all three if your organisation spans several layers.
| Page | Audience | What it covers |
|---|---|---|
| How to Reduce my OdataLink Usage - For Everyday Users | Accountants, bookkeepers, business users | Refresh frequency, model settings (Cut-off Date, Default Date Range, Polling), monitoring |
| How to Reduce my OdataLink Usage - For Advanced Users | Power BI, Fabric, and Excel report developers (Power Query / M) | Query folding, OData $filter operators, $select projection, incremental refresh
|
| How to Reduce my OdataLink Usage - For Power Users | Data engineering teams (SSIS, ADF, Fabric Data Pipelines, Fabric Dataflow Legacy/Gen2) | Stage-once-serve-many, watermarked incremental loads, historical backfill strategy, scheduling discipline, end-to-end cost monitoring |
Tip: Start with the Everyday Users page regardless of your role. The model-level settings described there apply to every refresh, automatic or manual, report or pipeline, and deliver the largest reduction with the least amount of changes.
Quick checklist
A cross-cutting checklist drawn from all three pages. Detailed steps and tool-specific guidance are in the linked pages above.
Everyone:
- [ ] Refresh schedule matches how often the report is actually used
- [ ] Weekend / out-of-hours refreshes disabled where not needed
- [ ] Archive Type set to Live (with Archive) on the model
- [ ] Cut-off Date or rolling window configured
- [ ] Default Date Range configured to match your reporting window
- [ ] Polling enabled
Advanced Users:
- [ ] Date filters applied at the OData navigation step (not post-download)
- [ ] Query folding verified via View Native Query
- [ ]
$selectused to limit columns retrieved - [ ] Incremental refresh enabled for time-series datasets
Azure/Fabric/Data Pipeline developers:
- [ ] OdataLink data staged once into SQL / Lakehouse / Warehouse / Dataflow Gen2; downstream consumers read from the staging layer
- [ ] All recurring loads use
$filterwith a high-watermark column - [ ]
$selectused on every entity, including the watermark column - [ ] Full-table backfills are one-time, not recurring
- [ ] Pipeline schedules staggered and aligned with the business cycle
- [ ] Reference and transactional data refreshed on separate cadences
- [ ] Both OdataLink egress and Azure / Fabric compute cost monitored
Need help?
If you have followed the steps above and your usage is still high, contact OdataLink support. We can review your model configuration, refresh patterns, and pipeline design, and recommend further optimisations to bring your account usage within limits.