Power Query Recipes
Master Your Data in
Excel & Power BI
© www.powerquery.academy
About Power Query Recipes
These cards are a set of ready-to-use Power Query Recipes to lead you step-by-step through a
variety of data transformations and techniques. These cards are a useful reference where you
can look up the exact steps needed for a particular data transformation pattern. We hope you
find them as handy as we do ourselves!
Using the Recipe Cards
The Table of Contents and the recipe cards are fully linked so you can easily navigate between
them. Clicking on the recipe number or name on the Table of Contents will take you directly to
that recipe. Likewise, if a recipe card references another recipe, clicking that reference will take
you directly to the referenced recipe.
Many of the recipes are also linked to corresponding videos in our Power
Query Academy course and are indicated with the video icon shown here:
If you are enrolled in the full Power Query Academy course (not in the free trial or any of
our other courses), clicking the video icon will take you to that video on the course site. Please
note that you will first need to login to the course site before being able to watch the video.
If you are on the ongoing subscription of these cards, we will let you know when new cards
have been added to the set you so can download your updated copy.
© www.powerquery.academy
Power Query Recipes – Table of Contents
00. Best Practice Architectures
0.100.1902
Files as a Data Source
n/a
0.110.1902
Creating a Staging Query
n/a
0.120.1902
Modifying Query Load Destinations
n/a
10. Splitting Data
10.100.1809
Split Records into Columns
Module 3, Video 3
10.105.1809
Split Records into Rows
Module 3, Video 4
20. Reshaping Data
Look for the
Pattern
Difficulty scale:
20.100.1809
Pivoting Data
Module 3, Video 2
20.105.1809
Unpivoting Data
Module 3, Video 1
20.110.1809
Transposing Data
Module 9, Video 3
20.115.1809
Unpivot Subcategorized Data
Module 9, Video 3
20.116.1809
Unpivot Subcategories – Big Data
Module 9, Video 3
20.120.1809
Pivoting Stacked Data
Module 9, Video 2
TOC: 00.001.1905
No
Code
Lo
Code
Pro
Code
© www.powerquery.academy
Power Query Recipes – Table of Contents
30. Combining Data
30.100.1809
7 Ways to Join Your Data
Module 5, Video 2
30.105.1809
Exact Match
Module 5, Video 1
30.110.1809
Left Outer Join
Module 5, Video 2
30.111.1809
Right Outer Join
Module 5, Video 2
30.112.1809
Full Outer Join
Module 5, Video 2
30.113.1809
Inner Join
Module 5, Video 2
30.114.1809
Left Anti Join
Module 5, Video 2
30.115.1809
Right Anti Join
Module 5, Video 2
30.116.1810
Full Anti Join
Module 5, Video 2
30.120.1809
Cartesian Product (Many to Many Join)
Module 5, Video 3
30.125.1809
Approximate Match
Module 5, Video 4
35. Appending Data
Look for the
Pattern
Difficulty scale:
35.400.1901
Appending Data Recipe Navigation
n/a
35.410.1811
Local/Network Drive “FilesList” Query
Module 4, Video 4
35.412.1901
SharePoint/Office 365 Groups “FilesList”
Query
n/a
TOC: 00.002.1905
No
Code
Lo
Code
Pro
Code
© www.powerquery.academy
Power Query Recipes – Table of Contents
35. Appending Data Continued…
35.414.1901
OneDrive for Business “FilesList” Query
n/a
35.420.1901
Combine All “Flat” Files in a Folder
Module 4, Video 4
35.500.1905
Appending a Named Worksheet or Table from
Multiple Excel files
n/a
35.510.1905
Appending Contents of xth Table or Worksheet
from Multiple Excel files
n/a
40. Conditional Logic
40.100.1809
Extract to New Column Based on Prior Row
Module 11, Video 6
40.105.1809
Extract to New Column Based on Next Row
Module 11, Video 6
50.100.1809
Ordinal Ranking
n/a
50.105.1905
Standard Competition Ranking
n/a
50.110.1905
Modified Competition Ranking
n/a
50.115.1905
Dense Ranking
n/a
50.120.1905
Fractional Ranking
n/a
50.125.1903
Number Grouped Rows
n/a
50. Ranking
Look for the
Pattern
Difficulty scale:
TOC: 00.003.1905
No
Code
Lo
Code
Pro
Code
© www.powerquery.academy
Power Query Recipes – Table of Contents
60. Date & Time Techniques
60.100.1809
Dynamic Calendar Boundaries
Module 15, Video 2
60.105.1809
Dynamic Calendar: Start Date to End Date
Module 15, Video 2
60.110.1809
Dynamic Calendar: Start Date + x Days
Module 15, Video 3
60.900.1903
Convert a Unix Timestamp to a Valid Date
n/a
60.910.1903
Convert UTC DateTime to Local DateTime
n/a
60.920.1903
Switching DateTimes Zones
n/a
70. Miscellaneous
70.100.1809
Look for the
Pattern
Difficulty scale:
Add a Random Number to all Table Rows
n/a
TOC: 00.004.1905
No
Code
Lo
Code
Pro
Code
© www.powerquery.academy
0. Best Practice
Architectures
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Architecture Recipes: Options when using Files as a Data Source
Individual Files
Save the new
data file with a
different name
The Benefits
All Files in a Folder
Save the new
data file over
the original
Save the new data file
under a new name
(in the same folder)
Open the
Excel Workbook/
Power BI Report
Open the Excel
Workbook/
Power BI Report
Build your solution once and then
re-use your existing business
intelligence reports and logic!
* If you are using Power BI, you may
be able to schedule the refresh
Edit the Query
Update the
Source path to
the new file
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Close the
Power Query
Editor
Refresh
the Data*
Update the Visuals
Recipe:
For more data patterns check out our Academy at:
© www.powerquery.academy
0.100.1902
Pattern Difficulty
Master Your Data in Excel & Power BI
Architecture Recipes: Creating a Staging Query
Excel
Power BI Desktop
Step 1:
Prepare the Source Data
Connect to your Data source
Perform any desired transformations
Name your query
Step 2:
Load as a Staging query
Right click the Query name
Uncheck Enable Load
Step 1:
Prepare the Source Data
Connect to your Data source
Perform any desired transformations
Name your query
Step 2:
Load as a Staging query
Go to Home → Close & Load → Close & Load To…
Choose Only Create Connection
Uncheck “Add this data to the Data Model”
Staging queries do not consume any processor
or RAM until they are called by another query.
They can be very helpful in order to act as a
starting point for other queries!
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe:
For more data patterns check out our Academy at:
© www.powerquery.academy
0.110.1902
Master Your Data in Excel & Power BI
Pattern Difficulty
Architecture Recipes: Modifying Query Load Destinations
Excel
Power BI Desktop
Step 1: Display the Queries & Connections Pane
Excel 2010/2013
Excel 2016
Power Query → Show Pane
Data → Show Queries
Excel 2019/365
Data → Queries & Connections
Step 2: Modify the Load Destination
Go to Home → Edit Queries
Right click the Query name
Click Close & Apply
Go to Home → Refresh
Right click the desired query → Load To…
Modify the load behavior as desired
Excel Tips!
1) If you accidentally loaded a query to a table, you can
convert it to a Connection Only query by simply
deleting the worksheet!
2) Change your default Load Behaviour by modifying
the Query Options found here:
Excel 2010/2013
Excel 2016
Excel 2019/365
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Power Query → Options
Data → New Query → Query Options
Data → Get Data → Query Options
Recipe:
For more data patterns check out our Academy at:
© www.powerquery.academy
0.120.1902
10. Splitting Data
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
Split Records Into Columns Recipe
Right click the Values column
Choose to Split Column → by Delimiter
Original
Choose appropriate Delimiter
Click OK
Caution
Where this works well:
• Desired output is Pivoted
• Number of elements never changes
It is a good idea to remove the
auto-generated Changed Type
step in case the number of
columns changes on update!
Did you know?
Output
You can use the Unpivot Other Columns command
in order to unpivot the results of this table
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 10.100.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Split Records Into Rows Recipe
Right click the Values column
Original
Choose to Split Column → by Delimiter
Choose appropriate Delimiter
Click Advanced → Split to Rows → OK
Where this works well:
• Desired output is Unpivoted
• Number of elements may change
Desired Output
Did you know?
You can pivot the Records column to ensure
that you’ll always get all columns if the number
of elements per record may change
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 10.105.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
20. Reshaping Data
© www.powerquery.academy
Master Your Data in Excel & Power BI
Headers
Fact
Pattern Difficulty
Pivoting Data Recipe
Select the [Headers] column
Go to Transform → Pivot Column
Change the Values drop-down to the [Fact] column
Click OK
Source Table
Headers
Output Table
Fact
Expanding the Advanced Options will allow
you to change the aggregation function
used from SUM to something different
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 20.100.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Unpivoting Data Recipe
Pivoted Data
Prepare Source table
Remove the Changed Type Step *
Filter out Total and Subtotal rows
Delete any Total and Subtotal columns
Unpivot the Data
Pivoted Data (Transposed)
Select all columns to keep as is
Right click → Unpivot Other Columns
Rename the [Attribute] column
Rename the [Value] column
Set data types
Unpivoted Data
* Only required if your source data’s column
names will change
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 20.105.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Transposing Data Recipe
Prepare the Source table
Filter out Total and Subtotal rows 1
Delete any Total and Subtotal columns 1
Transpose the Data
Data View
Demote headers if you wish to keep them 2
Go to Transform → Transpose
Promote First Row to Headers
Set data types
1. We recommend removing all totals and
subtotals as they can easily be rebuilt via
a PivotTable or PowerBI Matrix
2. To Demote headers, go to
Home → Use Headers as First Row → Use
First Row as Headers
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Alternate Data View
Recipe: 20.110.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Source Table
Unpivot Subcategorized Data Recipe
1. Demote the headers (if required)
2. Transpose the data
3. Fill up/down columns where needed
4. Merge original “headers” with a delimiter *
5. Transpose the data (back to original form)
6. For columns where row 1 shows only the delimiter,
replace values with text for column headers
7. Promote row 1 to headers
Output Table
8. Perform any cleanup that can be done before unpivoting
9. Unpivot columns
10.Split the [Attribute] column by the delimiter
11.Perform final cleanup
* Pick a delimiter that doesn’t exist elsewhere in
your data!
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 20.115.1902
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Unpivot Subcategories – Big Data Recipe
This recipe refers to the steps used in the regular
Unpivot Subcategorized Data recipe (Recipe 20.115.x)
LEGEND:
Connection Only Query
Headers
Query
Load to Table/Model
Data
Source
RawData
Query
Query Steps
Reference RawData
Keep top x rows
Recipe steps 2 to 6
Query Steps
Connect to Data
Recipe step 1
Set data types
Output
Query
Data
Query
Query Steps
Reference Headers Query
Append Data Query
Recipe steps 7 to end
Query Steps
Reference RawData
Remove top x rows
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 20.116.1902
For more data patterns check out our Academy at:
©www.powerquery.academy
www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Pivoting Stacked Data Recipe
Source Table
Add a columns for transaction and line IDs
Add Column → Index Column → From 0
Select [Index] → Add Column → Standard → Integer-Divide*
Select [Index] → Transform → Standard → Modulo*
Pivot the data
Select [Index] → Transform → Pivot Column
Set the Values field to your data column
Go Advanced and “Don’t Aggregate”
Right click the [Integer-Divide] column → Remove
Rename the newly created columns
Output Table
Set your data types
* Modulo and Integer Divide always
target the number representing the
1st row of the 2nd transaction
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 20.120.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
30. Combining Data
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
7 Ways to Join your Data
Inner Join
Only records
which match
(no red/yellow)
Recipe 30.113.x
Full Outer Join
All records
from both
Recipe 30.112.x
Left Outer Join
Records with
no match
(red + yellow)
Recipe 30.116.x
Right Outer Join
All records
from left
All records
from right
(no yellow)
Recipe 30.110.x
(no red)
Recipe 30.111.x
Left Anti Join
Right Anti Join
Mismatches
from left
Mismatches
from right
(only red)
Recipe 30.114.x
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Full Anti Join
(only yellow)
Recipe 30.115.x
Recipe: 30.100.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Source Table
Pattern Difficulty
Exact Match Recipe
Prepare Source table
Set data types
Load as Connection Only
Key
Column
Key
Column
Prepare Lookup table
Set data types
Right click [Key] column → Remove Duplicates
Load as Connection Only
Perform the match
Reference the Source table
Go to Home → Merge → Lookup table
Select the [Key] column on both tables
Use a Left Outer Join
Lookup Table
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Expand the new column
Uncheck the [Key] column and prefix option
Recipe: 30.105.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
Left Outer Join:
All records from Left (top) &
Matches from Right (bottom)
Source Table
Left Outer Join Recipe
Pink records have no
match in Lookup so are
included in result
For each table
Create a staging query (load as Connection Only)
Merge the Data
Reference the Source table
Go to Home → Merge → Lookup table
Key
Columns
Select the [Key] column(s) on both tables*
Use a Left Outer Join
Expand the Data
Key
Columns
Yellow records have no
match in Source so are
excluded from result
Expand the new column
Uncheck the [Key] column(s)
Consider unchecking Preface option
* Use multiple columns for your
[Key] column by holding down
the CTRL key while selecting them
Lookup Table
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 30.110.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
Right Outer Join:
All records from Right (bottom)
& Matches from Left (top)
Source Table
Right Outer Join Recipe
Pink records have no
match in Lookup so are
excluded from result
For each table
Create a staging query (load as Connection Only)
Merge the Data
Reference the Source table
Go to Home → Merge → Lookup table
Key
Columns
Select the [Key] column(s) on both tables*
Use a Right Outer Join
Expand the Data
Key
Columns
Yellow records have no
match in Source so are
included in result
Expand the new column
Uncheck the [Key] column(s)
Consider unchecking Preface option
* Use multiple columns for your
[Key] column by holding down
the CTRL key while selecting them
Lookup Table
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 30.111.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
Full Outer Join:
All records from both tables
Source Table
Full Outer Join Recipe
Pink records
are included in
result
Yellow records
are included in
result
For each table
Create a staging query (load as Connection Only)
Merge the Data
Reference the Source table
Go to Home → Merge → Lookup table
Key
Columns
Select the [Key] column(s) on both tables*
Use a Full Outer Join
Key
Columns
Expand the Data
Expand the new column
Uncheck the [Key] column(s)
[Account] & [Dept]
shown here are from
the Source table only
Consider unchecking Preface option
* Use multiple columns for your
[Key] column by holding down
the CTRL key while selecting them
Lookup Table
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 30.112.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
Inner Join:
Only records with matches
Source Table
Inner Join Recipe
Pink records have no match in
Lookup so are excluded from result
For each table
Create a staging query (load as Connection Only)
Merge the Data
Reference the Source table
Go to Home → Merge → Lookup table
Key
Columns
Select the [Key] column(s) on both tables*
Use an Inner Join
Key
Columns
Expand the Data
Yellow records have no
match in Source so are
excluded from result
Expand the new column
Uncheck the [Key] column(s)
Consider unchecking Preface option
* Use multiple columns for your [Key]
column by holding down the CTRL
key while selecting them
Lookup Table
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 30.113.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
Left Anti Join:
Only records from Left (top)
with no match in Right (bottom)
Source Table
Left Anti Join Recipe
Pink records have no match in
Lookup so are included in result
Key
Columns
Key
Columns
For each table
Create a staging query (load as Connection Only)
Merge the Data
Reference the Source table
Go to Home → Merge → Lookup table
All records that match
between the two tables
are excluded from result
Yellow records have no
match in Source and are
excluded from result
Select the [Key] column(s) on both tables*
Use a Left Anti Join
Expand the Data
Expand the new column
Uncheck the [Key] column(s)
Consider unchecking Preface option
* Use multiple columns for your
[Key] column by holding down
the CTRL key while selecting them
Lookup Table
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 30.114.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
Right Anti Join:
Only records from Right (bottom)
with no match in Left (top)
Source Table
Right Anti Join Recipe
Yellow records have no match in
Source so are included in result
Key
Columns
Key
Columns
For each table
Create a staging query (load as Connection Only)
Merge the Data
Reference the Source table
Go to Home → Merge → Lookup table
All records that match
between the two tables
are excluded from result
Pink records have no
match in Lookup and
are excluded from result
Select the [Key] column(s) on both tables*
Use a Right Anti Join
Expand the Data
Remove the [Key] column(s)
Expand the new column
Consider unchecking Preface option
* Use multiple columns for your
[Key] column by holding down
the CTRL key while selecting them
Lookup Table
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 30.115.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Source Table
Pattern Difficulty
Full Anti Join:
Show only records that do not
have matches in the other table
Only items where the Key column
doesn’t have a match will be
displayed
Full Anti Join Recipe
Create Left Anti-Join query* (Recipe 30.116.x)
Create Right Anti-Join query* (Recipe 30.117.x)
Expand all fields from joined [Key] column†
Key
Column
Key
Column
This join is not available
in the user interface and
must be created
manually!
Load as staging queries (Connection Only)
Reference Left Anti-Join query
Go to Home → Append → Right Anti-Join query
Load to your destination
* Ensure names of columns are identical in both queries
†
If both tables contain the same field names, rename
the Join column then expand it with the Preface option
checked
Lookup Table
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 30.116.1810
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Cartesian Product Recipe
(Many to Many Joins)
Table 1
For each table to be merged:
Create a staging query
Create a merge query
Table 2
Reference Table1
Go to Add Column → Custom Column
Name: Leave as “Custom”
Formula:=Table2
Expand the new column*
The Result
* Be sure to uncheck the Preface option
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 30.120.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Source Table
Approximate Match Recipe
Prepare Source table
Set data types
Key
Match
Match
Return
Prepare Lookup table
Output Table
Rename [Match] column to match Source table
Set data types
Perform the match
Reference the Source table
Go to Home → Append → Lookup table
Filter the [Match] column → Sort Ascending
Filter the [Key] column → Sort Ascending
Right click the [Return] column → Fill → Down
Filter the [Key] column → uncheck null
Key
Match
Return
Lookup Table
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 30.125.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
35. Appending Data
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
Appending Data Recipe Navigation
Create a “FilesList”
query to read
from folder
Local / Network Drive:
SharePoint / O365:
OneDrive for Business:
Create a “Master”
query to
append files
Reference the
FilesList query
Modify the
Sample Transform
Perform transforms that
MUST be done BEFORE
the files are appended
Clean up the
“Master” query
Recipe 35.410.x
Recipe 35.412.x
Recipe 35.414.x
Perform transforms that
can be done AFTER
the files are appended
Appending Data Recipes
Flat Files (.txt, .csv, .prn)
Excel files:
Based on named sheet
Based on xth sheet
Based on all sheets
35.420.x
available
in future
version
Benefits of Using a “FilesList” Query
- An easy entry point to review and sort the files list
- The file path gets hard-coded into the solution once (easier to update in future)
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 35.400.1901
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Get Files
From Folder
Rename
“FilesList”
Local/Network Drive “FilesList” Query Recipe
Step 1: Connect to the Folder
Get Data → From File → From Folder
Browse to the folder you need
Choose to Edit/Transform the Data
Step 2: Filter and Future-Proof the Files List
Optional: Filter [Folder Path] to the desired subfolder
Right click the Extension column → Transform → lowercase
Filter the Extension column → Text Filters → Equals
Type in your desired file type*
Filter the Name column → Does Not Begin With → ~
Perform any additional sorting, filtering or modifications**
Filter to
subfolders as
required
Filter to the
specific file type
Pattern Difficulty
Step 3: Finalize the Query
Load the query as Connection Only
Tips and Tricks
Load “FilesList”
as “Connection”
* If you need more than one type, consider using Contains or Does Not Contain for your filter
** Can be useful if you need to use a specific file as your sample during the combine operation
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 35.410.1811
For more data patterns check out our Academy at:
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
SharePoint/Office 365 Groups “FilesList” Query Recipe
Tips and Tricks
* If you need more than
one file type, consider
using Contains or Does
Not Contain for your filter
** This step may be useful
if you need to use a specific
file as your sample during
the combine operation
Step 1: Connect to the Folder
Get Data → From File → From SharePoint Folder
Enter the URL to the root of your site (not a library or folder)†
Chose to Edit/Transform the data
Step 2: Filter and Future-Proof the Files List
Filter the [Folder Path] column to the desired library or folder
Right click the [Extension] column → Transform → lowercase
Filter the [Extension] column → Text Filters → Equals
Type in your desired file type*
Filter the [Name] column → Does Not Begin With → ~
Perform any additional sorting, filtering or modifications**
Load the query as Connection Only
† Locating your SharePoint Site URL:
Find the second “/” to the left of the word “Forms” and keep everything to the left:
https://<SharePointDomain>/sites/projects/rockets/Forms/AllItems.aspx
<----------- connect to this ----------->|<---- ignore all this ---->
Get Files From
Sharepoint
Folder
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Rename
“FilesList”
Filter to library
or folder as
required
Filter to the
specific file type
Load “FilesList”
as “Connection”
Recipe: 35.412.1901
For more data patterns check out our Academy at:
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
OneDrive for Business “FilesList” Query Recipe
Tips and Tricks
* If you need more than
one file type, consider
using Contains or Does
Not Contain for your filter
** This step may be useful
if you need to use a specific
file as your sample during
the combine operation
Step 1:
Connect to the Folder
Get Data → From File → From SharePoint Folder
Enter the URL to the root of your OneDrive for Business site†
Chose to Edit/Transform the data
Step 2:
Filter and Future-Proof the Files List
Filter the [Folder Path] column to the desired library or folder
Right click the [Extension] column → Transform → lowercase
Filter the [Extension] column → Text Filters → Equals
Type in your desired file type*
Filter the [Name] column → Does Not Begin With → ~
Perform any additional sorting, filtering or modifications**
Load the query as Connection Only
† Connect to the root of your OneDrive for Business site as follows:
https://<tenant>-my.sharepoint.com/personal/<email>
where the . and @ signs in your email are replaced with the _ character.
(The easiest way to check is to login to OneDrive for Business in the web and inspect the URL.)
Get Files From
Sharepoint
Folder
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Rename
“FilesList”
Filter to library
or folder as
required
Filter to the
specific file type
Load “FilesList”
as “Connection”
Recipe: 35.414.1901
For more data patterns check out our Academy at:
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
What is a “Flat” File?
It’s a file with no complex metadata such as a *.TXT, *.CSV, *.PRN. Can you
open it in Notepad and see the data? Chances are good that it’s a flat file!
Combine All “Flat” Files in a Folder
Step 1:
Create a FilesList query
Create the FilesList query indicated by Recipe 35.400.x
Step 2:
Create the Master query
Reference the FilesList Query
Make sure to rename the query BEFORE combining!
Click the  symbol at the top of the [Content] column
Choose the File for your sample → click OK
This will create 4 new queries!
Step 3:
Modify the Transform Sample from Master
Perform modifications required before the files are appended
Return to the Master query
Step 4:
These queries
are created
automatically
when you
combine files!
Finalize the Master query
Delete the Changed Type step to clear any errors
Perform any additional cleanup†
Define the data types for each column
Load the query to your desired destination
Power Query Academy
Subscribers: Click here to
see this recipe in action!
† Tip:
Did you lose [FileName]
or other needed columns
upon combining?
Click the gear next to the
“Removed Other
Columns1” step to bring
them back!
Recipe: 35.420.1901
For more data patterns check out our Academy at:
© www.powerquery.academy
Pattern Difficulty
Master Your Data in Excel & Power BI
Appending contents of a named Worksheet
(or Table) from Multiple Excel Files
Step 1:
Create a FilesList query
Create the FilesList query indicated by Recipe 35.400.x
Step 2:
Create the Master query
Reference the FilesList Query
Make sure to rename the query BEFORE combining!
Click the  symbol at the top of the [Content] column
These queries
are created
automatically
when you
combine files!
Choose the File for your sample → click OK
Select the Worksheet (or Table) you wish from each workbook
This will create 4 new queries!
Step 3:
Modify the Transform Sample from Master
Perform modifications required before the files are appended
Return to the Master query
Step 4:
Finalize the Master query
Delete the Changed Type step to clear any errors
Perform any additional cleanup†
Define the data types for each column
Load the query to your desired destination
Power Query Academy
Subscribers: Click here to
see this recipe in action!
† Tip:
Did you lose [FileName]
or other needed columns
upon combining?
Click the gear next to the
“Removed Other
Columns1” step to bring
them back!
Recipe: 35.500.1905
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Appending only contents of xth Worksheet
(or Table) from Multiple Excel Files
Step 1:
Step 2:
Create a FilesList query per Recipe 35.400.x
Create the Master query
Reference the FilesList Query
Make sure to rename the query BEFORE combining!
Pattern Difficulty
These queries
are created
automatically
when you
combine files!
Click the  symbol at the top of the [Content] column
Select the Folder icon (not a worksheet) → Click OK
Step 3:
Modify the Transform Sample from Master
Filter the [Kind] column to either Sheet or Table
Go to Add Column → Add Index Column → From 1
Filter the [Index] column to the item you wish to keep
To preserve the names of the
Tables or Worksheets as well
as the data, see:
Recipe 35.520.x (Tables)
Right click the [Data] column → Remove Other columns
Recipe 35.530.x (Worksheets)
Click the Table keyword to drill in
† Tip:
Perform modifications required before the files are appended
Did you lose [FileName]
Return to the Master query
or other needed columns
Step 4:
Finalize the Master query
upon combining?
Delete the Changed Type step to clear any errors
Click the gear next to the
†
“Removed Other
Perform any additional cleanup
Columns1” step to bring
Define the data types for each column
them back!
Load the query to your desired destination
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 35.510.1905
For more data patterns check out our Academy at:
© www.powerquery.academy
40. Conditional Logic
© www.powerquery.academy
Master Your Data in Excel & Power BI
Extract to New Column Based
on Data in Prior Row Recipe
Target
Pattern Difficulty
Compare
Preparation:
Perform any possible preliminary cleanup
Add [Index] column: Add Columns→Index Column→From 0
Pattern
Add [Index.1] column: Add Columns→Index Column→From 1
Extract Data Into New Column:
Go to Home → Merge Queries As New → Merge against itself
Match [Index] (top) against [Index.1] (bottom)
Expand the [Compare] column only, creating [Compare.1]
Sort by the [Index] column (ascending)
Create a Conditional Column:
=if [Compare.1]= "pattern" then [Target] else null
Extract these
Based on prior
row values
Right click the new column → Fill → up/down (as needed)
Filter the [Compare.1] column to remove the pattern
Remove the [Index], [Index.1] and [Compare.1] columns
And put it into
a new column:
Finish cleanup
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 40.100.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Extract to New Column Based
on Data in Next Row Recipe
Target
Pattern Difficulty
Compare
Preparation:
Perform any possible preliminary cleanup
Add [Index] column: Add Columns→Index Column→From 0
Add [Index.1] column: Add Columns→Index Column→From 1
Pattern
Extract Data Into New Column:
Go to Home → Merge Queries As New → Merge against itself
Match [Index.1] (top) against [Index] (bottom)
Expand the [Compare] column only, creating [Compare.1]
Sort by the [Index] column (ascending)
Create a Conditional Column:
=if [Compare.1]= "pattern" then [Target] else null
Extract these
Based on prior
row values
Right click the new column → Fill → up/down (as needed)
Filter the [Compare.1] column to remove the pattern
Remove the [Index], [Index.1] and [Compare.1] columns
And put it into
a new column:
Finish cleanup
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 40.105.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
50. Ranking
© www.powerquery.academy
Master Your Data in Excel & Power BI
An ordinal rank creates a
unique number for each
row and never breaks ties
Pattern Difficulty
Source Table
Ordinal Ranking Recipe
Initial Preparation
Perform any possible preliminary cleanup
Set data types
Sort the numeric column → Descending
Sort any other columns to order ties
Ranked Table
Add Ranking
Go to Add Column → Index Column → From 1
Rename [Index] to Rank
Learn about ranking methods:
https://en.wikipedia.org/wiki/Ranking
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 50.100.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Standard Competition
Ranking Recipe
Initial Preparation
Perform any possible preliminary cleanup
Set data types
Sort the [Fact] column → Descending
Sort any [Dimension] column(s) to order ties
Go to Add Column → Index Column → From 1
Group the Data
Go to Transform → Group → Advanced
Group by the [Fact] column
Add new column names as follows:
Pattern Difficulty
Standard Competition Ranking
records ties, leaving gaps in the
ranking numbers AFTER sets of
equal-ranking items
Source Table
Ranked Table
Rank → Min → [Index]
Data → All Rows
Expand the [Dimension] field from the [Data] column
Re-order the columns if desired
Learn about ranking methods:
https://en.wikipedia.org/wiki/Ranking
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Dimension
Fact
Recipe: 50.105.1905
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Modified Competition
Ranking Recipe
Initial Preparation
Perform any possible preliminary cleanup
Set data types
Sort the [Fact] column → Descending
Sort any [Dimension] column(s) to order ties
Go to Add Column → Index Column → From 1
Group the Data
Pattern Difficulty
Modified Competition Ranking
records ties, leaving gaps in the
ranking numbers BEFORE sets of
equal-ranking items
Source Table
Ranked Table
Go to Transform → Group → Advanced
Group by the [Fact] column
Add new column names as follows:
Rank → Max → [Index]
Data → All Rows
Expand the [Dimension] field from the [Data] column
Re-order the columns if desired
Learn about ranking methods:
https://en.wikipedia.org/wiki/Ranking
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Dimension
Fact
Recipe: 50.110.1905
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Dense Ranking Recipe
Initial Preparation
Perform any possible preliminary cleanup
Set data types
Sort the [Fact] column → Descending
Sort any [Dimension] column(s) to order ties
Pattern Difficulty
Dense Ranking records ties,
but does not leave gaps in the
ranking numbers before or
after sets of equal-ranking
items
Group the Data
Go to Transform → Group → Advanced
Group by the [Fact] column
Add a new column name as follows:
Source Table
Ranked Table
Data → All Rows
Sort the [Fact] column → Descending
Go to Add Column → Index Column → From 1
Expand the [Dimension] field from the [Data] column
Rename the [Index] column to “Rank”
Re-order the columns if desired
Learn about ranking methods:
https://en.wikipedia.org/wiki/Ranking
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Dimension
Fact
Recipe: 50.115.1905
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Fractional Ranking Recipe
Initial Preparation
Sort the [Fact] column → Descending
Sort any [Dimension] column(s) to order ties
Go to Add Column → Index Column → From 1
Fractional Ranking records ties,
calculating them as the mean
of the values that would be
generated using ordinal ranking
Group the Data
Go to Transform → Group → Advanced
Group by the [Fact] column with these new columns:
Std → Min → [Index]
Source Table
Ranked Table
Mod → Max → [Index]
Data → All Rows
Go to Add Column → Custom Column
Name:
Rank
Formula: =([Std]+[Mod])/2
Remove the [Std] and [Mod] columns
Expand the [Dimension] field from the [Data] column
Re-order the columns if desired
Dimension
Fact
Learn about ranking methods:
https://en.wikipedia.org/wiki/Ranking
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 50.120.1905
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Number Grouped Rows Recipe
Pattern Difficulty
Source Table
Initial Preparation
Sort any [Dimension] column(s) to group the data
Sort the [Fact] column → Descending
Re-apply the column data types
Group the Data
Go to Transform → Group → Advanced
Group by the [Dimension] column(s)
Add a new column called “Data” → All Rows
Dimension
Add Row Numbers
Fact
Table With Numbered Rows
Go to Add Column → Custom Column
Name:
Custom
Formula: =Table.AddIndexColumn([Data], "Index", 1, 1)
Right click the [Custom] column → Remove Other Columns
Expand all columns from [Custom]
Re-order the columns if desired
DBAs know this output as using SQL’s
ROW_NUMBER with PARTITION
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 50.125.1903
For more data patterns check out our Academy at:
© www.powerquery.academy
60. Date & Time Techniques
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Dynamic Calendar Boundaries Recipe
Determine the StartDate
Determine the EndDate
Reference table with earliest date
Reference table with latest date
Right click [Date]→ Remove Other Columns
Right click [Date]→ Remove Other Columns
Filter [Date] → Date Filters → Is Earliest
Filter [Date] → Date Filters → Is Latest
Right click [Date] → Remove duplicates
Right click [Date] → Remove duplicates
Change the data type to a Date*
Change the data type to a Date*
Transform → Date → Year → Start of Year
Transform → Date → Year → End of Year
Right click the date cell→ Drill down †
Right click the date cell→ Drill down†
Call query StartDate (no spaces)
Call query EndDate (no spaces)
Load as Connection Only
Load as Connection Only
Continue with your preferred Dynamic Calendar Recipe
* This step is important
if you are not forcing
to start/end of period
Power Query Academy
Subscribers: Click here to
see this recipe in action!
†
StartDate
Ensure that you click the cell to drill down, not the header
of the table. You need your new formula to read as follows:
=#"<previous step name>"{0}[Date Column Name]
Recipe: 60.100.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Dynamic Calendar Recipe:
From Start Date to End Date
Determine the Calendar boundaries
Create StartDate query (Recipe 60.100.x)
Create EndDate query (Recipe 60.100.x)
Create a Blank Query
Given only a
StartDate and
an EndDate…
Enter the following in the formula bar:
={Number.From(StartDate)..Number.From(EndDate)}
Go to List Tools → To Table → OK
Set [Column1] to a Date data type
Rename [Column1] to Date
Rename the table to Calendar
Add New Columns
Fully Dynamic ‘Calendar’ Table
1. Select the [Date] column
2. Go to Add Column → Date → <the format to add>
3. Repeat Steps 1-2 as many times as needed
Load the Calendar to the intended destination
To shorten Month or Day names, go to
Transform → Extract → First Characters
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 60.105.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Dynamic Calendar Recipe:
Start Date + x Days
Determine the Calendar boundaries
Create StartDate query (Recipe 60.100.x)
Create a Days query or parameter (for number of days)*
Create a Blank Query
Enter the following in the formula bar:
=List.Dates(StartDate,Days,#duration(1,0,0,0))
Go to List Tools → To Table → OK
Set [Column1] to a Date data type
Rename [Column1] to Date
Rename the table to Calendar
Pattern Difficulty
* Creating a Days query or
parameter is optional. If
you choose not to, then just
substitute a number into
the formula in place of Days
Given only a
StartDate
and x Days…
Fully Dynamic ‘Calendar’ Table
Add New Columns
1. Select the [Date] column
2. Go to Add Column → Date → <the format to add>
3. Repeat Steps 1-2 as many times as needed
Load the Calendar to the intended destination
Power Query Academy
Subscribers: Click here to
see this recipe in action!
To shorten Month or Day names, go to
Transform → Extract → First Characters
Recipe: 60.110.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
epoch
Convert a Unix Timestamp to a Valid Date
Pattern Difficulty
Time (UTC)
Initial Preparation
Convert the [epoch] column to a Whole Number data type
Add a Column for UTC Time
Go to Add Column → Custom Column
Name:
Time (UTC)
Formula: =#datetime(1970, 1, 1, 0, 0, 0)
+ #duration(0, 0, 0, [epoch])
Set the [Time (UTC)] column to a DateTime data type
Right click the [epoch] column → Remove Column
Unix time (also known as POSIX time or UNIX Epoch time) displays
time as the number of seconds that have elapsed since 00:00:00
Thursday, 1 January 1970, minus leap seconds. All times are
tracked in Coordinated Universal Time (UTC).
Need to convert the UTC
time to another time zone?
Check out Recipe 60.910.x
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Recipe: 60.900.1903
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Pattern Difficulty
Convert UTC DateTime to Local DateTime Recipe
Step 1:
Add a UTC time zone (if your data has time zones, skip to Step 2)
Right click the [Original] column → Change Type → DateTime
Go to Add Column → Custom Column
Name: DateTimeLocal
Formula:DateTime.AddZone([Original],0)
Right click the [Original] column → Remove Column
Step 2:
Convert the [DateTimeLocal] column to your local time zone
Select the [DateTimeLocal] column → Transform → Time → Local Time
Right click the [DateTimeLocal] column → Change Type → <Date or DateTime as desired>
Original
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Original Converted to Local Zone
NOTE: This recipe is
Daylight Savings Time
aware! If you do not
want this volatility in
your transformation,
see Recipe 60.920.x to
lock in a time zone.
Local Date
Recipe: 60.910.1903
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data in Excel & Power BI
Switching DateTime Zones Recipe
Convert with 2A
Step 1: Create an “Offset” parameter
Go to Manage Parameters → New Parameter
Name:
Offset
Current Value: <enter number of hours time zones to offset>
Note that offset can be set as negative if required
Pattern Difficulty
Does your data
already show
time zones?
Convert with 2B
Step 2A: If your data DOES NOT already have a time zone
Right click the [Original] column → Change Type → DateTime
Go to Add Column → Custom Column
Name:
Formula:
NewZone
DateTime.AddZone([Original],Offset)
Right click the [Original] column → Remove Column
Step 2B: If your data ALREADY displays a time zone
Go to Add Column → Custom Column
Name:
NewZone
Formula: =DateTimeZone.SwitchZone([Original],Offset)
Power Query Academy
Subscribers: Click here to
see this recipe in action!
CAUTION!
These DateTimeZone
functions are NOT
Daylight Savings Time
aware! To correctly factor
in Daylight Savings Time
see Recipe 60.910.x.
Recipe: 60.920.1903
For more data patterns check out our Academy at:
© www.powerquery.academy
70. Miscellaneous
© www.powerquery.academy
Master Your Data in Excel & Power BI
Add a Random Number to all Table Rows
Pattern Difficulty
Source Table
Prepare your data table
Go to Add Column → Add Index column *
Go to Add Column → Add Custom column
Column Name:
RandomSeed
Formula:
=List.Random(1){0}
Go to Add Column → Add Index column *
Select [Index] and [Index.1] → Home → Remove Columns
Set [RandomSeed] to a Decimal data type
* Yes, this step is required, as it forces Power Query to
calculate and lock in the random seeds. Without it
you’ll get a single random number on all rows!
Power Query Academy
Subscribers: Click here to
see this recipe in action!
Add Random Numbers
to each table using
Power Query
Recipe: 70.100.1809
For more data patterns check out our Academy at:
© www.powerquery.academy
Master Your Data
www.powerquery.academy
© www.powerquery.academy