Reducing Data Usage and Optimising your OdataLink Feeds: Difference between revisions
mNo edit summary |
m fair use policy published, correcting the location of the policy doc on the website |
||
| (8 intermediate revisions by the same user not shown) | |||
| Line 7: | Line 7: | ||
* Improve refresh performance | * Improve refresh performance | ||
You can monitor your current usage at any time in your | You can monitor your current usage at any time in your '''OdataLink Admin Portal → Usage Stats'''. | ||
See [[How to track my OdataLink Usage]] for usage broken down by [[Data Files|Data file]], [[Models|Model]], and [[Users|User]]. | See [[How to track my OdataLink Usage]] for usage broken down by [[Data Files|Data file]], [[Models|Model]], and [[Users|User]]. | ||
| Line 14: | Line 14: | ||
== What is Usage == | == What is Usage == | ||
<onlyinclude> | |||
'''Usage''' is defined as the volume of data OdataLink retrieves from your connected [[Data Files|data files]] each month. | '''Usage''' is defined as the volume of data OdataLink retrieves from your connected [[Data Files|data files]] each month. | ||
This includes data retrieved by [[Microsoft Power BI|Power BI]], [[Microsoft Excel|Excel]], or [[OData Client Compatibility|other]] data pipeline and reporting tools, either via automatic scheduled refreshes or manual report refresh. | This includes data retrieved by [[Microsoft Power BI|Power BI]], [[Microsoft Excel|Excel]], or [[OData Client Compatibility|other]] data pipeline and reporting tools, either via automatic scheduled refreshes or manual report refresh. | ||
</onlyinclude> | |||
= Pick your Optimisation Layer | = Pick your Optimisation Layer = | ||
Optimisation techniques are grouped by the tooling you use. Pick the page that matches your role — or read all three if your organisation spans several layers. | Optimisation techniques are grouped by the tooling you use. Pick the page that matches your role — or read all three if your organisation spans several layers. | ||
| Line 28: | Line 29: | ||
| [[How to Reduce my OdataLink Usage - For Advanced Users]] || Power BI, Fabric, and Excel report developers (Power Query / M) || Query folding, OData <code>$filter</code> operators, <code>$select</code> projection, incremental refresh | | [[How to Reduce my OdataLink Usage - For Advanced Users]] || Power BI, Fabric, and Excel report developers (Power Query / M) || Query folding, OData <code>$filter</code> operators, <code>$select</code> projection, incremental refresh | ||
|- | |- | ||
| [[How to Reduce my OdataLink Usage - For Power Users]] || Data engineering teams (SSIS, ADF, Fabric Data Pipelines, Fabric Dataflow Gen2) || Stage-once-serve-many, watermarked incremental loads, historical backfill strategy, scheduling discipline, end-to-end cost monitoring | | [[How to Reduce my OdataLink Usage - For Power Users]] || Data engineering teams (SSIS, ADF, Fabric Data Pipelines, Fabric Dataflow Legacy/Gen2) || Stage-once-serve-many, watermarked incremental loads, historical backfill strategy, scheduling discipline, end-to-end cost monitoring | ||
|} | |} | ||
<span style="color:navy"><blockquote>'''Tip:''' Start with the [[How to Reduce my OdataLink Usage - For Everyday Users|Everyday Users]] page regardless of your role. The model-level settings described there apply to '''every''' refresh | <span style="color:navy"><blockquote>'''Tip:''' Start with the [[How to Reduce my OdataLink Usage - For Everyday Users|Everyday Users]] page regardless of your role. The model-level settings described there apply to '''every''' refresh, automatic or manual, report or pipeline, and deliver the largest reduction with the least amount of changes.</blockquote></span> | ||
<br /> | <br /> | ||
<!-- | |||
== For everyone (accountants, bookkeepers, business users) == | == For everyone (accountants, bookkeepers, business users) == | ||
| Line 57: | Line 58: | ||
<br> | <br> | ||
--> | |||
== Quick checklist == | == Quick checklist == | ||
A cross-cutting checklist drawn from all three pages. Detailed steps and tool-specific guidance are in the linked pages above. | A cross-cutting checklist drawn from all three pages. Detailed steps and tool-specific guidance are in the linked pages above. | ||
| Line 69: | Line 70: | ||
* [ ] Polling enabled | * [ ] Polling enabled | ||
''' | '''Advanced Users:''' | ||
* [ ] Date filters applied at the OData navigation step (not post-download) | * [ ] Date filters applied at the OData navigation step (not post-download) | ||
* [ ] Query folding verified via '''View Native Query''' | * [ ] Query folding verified via '''View Native Query''' | ||
| Line 75: | Line 76: | ||
* [ ] Incremental refresh enabled for time-series datasets | * [ ] Incremental refresh enabled for time-series datasets | ||
'''Data | '''Azure/Fabric/Data Pipeline developers:''' | ||
* [ ] OdataLink data staged once into SQL / Lakehouse / Warehouse / Dataflow Gen2; downstream consumers read from the staging layer | * [ ] OdataLink data staged once into SQL / Lakehouse / Warehouse / Dataflow Gen2; downstream consumers read from the staging layer | ||
* [ ] All recurring loads use <code>$filter</code> with a high-watermark column | * [ ] All recurring loads use <code>$filter</code> with a high-watermark column | ||
| Line 94: | Line 95: | ||
== Related Topics == | == Related Topics == | ||
* [https://odatalink.com/fair-use-policy OdataLink Fair Use Policy | * [https://odatalink.com/eula/fair-use-policy OdataLink Fair Use Policy] | ||
* [[How to track my OdataLink Usage]] | * [[How to track my OdataLink Usage]] | ||
* [[OData Feed URLs]] | * [[OData Feed URLs]] | ||
<!-- * [https://developer.xero.com/pricing Xero Developer Pricing] | <!-- * [https://developer.xero.com/pricing Xero Developer Pricing] | ||
* [https://developer.xero.com/faq/pricing-and-policy-updates Xero Pricing and Policy Updates FAQ]--> | * [https://developer.xero.com/faq/pricing-and-policy-updates Xero Pricing and Policy Updates FAQ]--> | ||