Reducing Data Usage and Optimising your OdataLink Feeds: Difference between revisions
mNo edit summary |
mNo edit summary |
||
| Line 18: | Line 18: | ||
This includes data retrieved by [[Microsoft Power BI|Power BI]], [[Microsoft Excel|Excel]], or [[OData Client Compatibility|other]] data pipeline and reporting tools, either via automatic scheduled refreshes or manual report refresh. | This includes data retrieved by [[Microsoft Power BI|Power BI]], [[Microsoft Excel|Excel]], or [[OData Client Compatibility|other]] data pipeline and reporting tools, either via automatic scheduled refreshes or manual report refresh. | ||
= Pick your Optimisation Layer | = Pick your Optimisation Layer = | ||
Optimisation techniques are grouped by the tooling you use. Pick the page that matches your role — or read all three if your organisation spans several layers. | Optimisation techniques are grouped by the tooling you use. Pick the page that matches your role — or read all three if your organisation spans several layers. | ||
| Line 28: | Line 28: | ||
| [[How to Reduce my OdataLink Usage - For Advanced Users]] || Power BI, Fabric, and Excel report developers (Power Query / M) || Query folding, OData <code>$filter</code> operators, <code>$select</code> projection, incremental refresh | | [[How to Reduce my OdataLink Usage - For Advanced Users]] || Power BI, Fabric, and Excel report developers (Power Query / M) || Query folding, OData <code>$filter</code> operators, <code>$select</code> projection, incremental refresh | ||
|- | |- | ||
| [[How to Reduce my OdataLink Usage - For Power Users]] || Data engineering teams (SSIS, ADF, Fabric Data Pipelines, Fabric Dataflow Gen2) || Stage-once-serve-many, watermarked incremental loads, historical backfill strategy, scheduling discipline, end-to-end cost monitoring | | [[How to Reduce my OdataLink Usage - For Power Users]] || Data engineering teams (SSIS, ADF, Fabric Data Pipelines, Fabric Dataflow Legacy/Gen2) || Stage-once-serve-many, watermarked incremental loads, historical backfill strategy, scheduling discipline, end-to-end cost monitoring | ||
|} | |} | ||
| Line 34: | Line 34: | ||
<br /> | <br /> | ||
<!-- | |||
== For everyone (accountants, bookkeepers, business users) == | == For everyone (accountants, bookkeepers, business users) == | ||
| Line 57: | Line 57: | ||
<br> | <br> | ||
--> | |||
== Quick checklist == | == Quick checklist == | ||
A cross-cutting checklist drawn from all three pages. Detailed steps and tool-specific guidance are in the linked pages above. | A cross-cutting checklist drawn from all three pages. Detailed steps and tool-specific guidance are in the linked pages above. | ||
| Line 69: | Line 69: | ||
* [ ] Polling enabled | * [ ] Polling enabled | ||
''' | '''Advanced Users:''' | ||
* [ ] Date filters applied at the OData navigation step (not post-download) | * [ ] Date filters applied at the OData navigation step (not post-download) | ||
* [ ] Query folding verified via '''View Native Query''' | * [ ] Query folding verified via '''View Native Query''' | ||
| Line 75: | Line 75: | ||
* [ ] Incremental refresh enabled for time-series datasets | * [ ] Incremental refresh enabled for time-series datasets | ||
'''Data | '''Azure/Fabric/Data Pipeline developers:''' | ||
* [ ] OdataLink data staged once into SQL / Lakehouse / Warehouse / Dataflow Gen2; downstream consumers read from the staging layer | * [ ] OdataLink data staged once into SQL / Lakehouse / Warehouse / Dataflow Gen2; downstream consumers read from the staging layer | ||
* [ ] All recurring loads use <code>$filter</code> with a high-watermark column | * [ ] All recurring loads use <code>$filter</code> with a high-watermark column | ||