User Profile
djclements
Silver Contributor
Joined Oct 21, 2023
User Widgets
Recent Discussions
Re: Cumulative Sum of Each Column or Row
I thought about this some more, but it still makes no sense to me how re-evaluating the same delayed calculations multiple times could possibly be faster than defining a variable once and reusing its results multiple times. After all, one of the supposed key benefits of LET is to improve performance by evaluating an expression once. I wondered if the performance boost we're seeing can be attributed solely to the benefits of LAMBDA, or if it's also an apparent flaw in the efficiency of LET, so I rewrote your Spiralλ function without LET or LAMBDA, repeating every instance of SEQUENCE (all of the i(), j(), x() and y() variables) within the nested IF statements: = LAMBDA(rings,[start_at], IF(ISOMITTED(start_at),1,start_at)+ IF( (ABS((SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0)))>ABS((SEQUENCE(rings*2+1,,rings,-1)*SEQUENCE(,rings*2+1,1,0))))*((SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0))>0), 4*(SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0))^2-3*(SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0))+(SEQUENCE(rings*2+1,,rings,-1)*SEQUENCE(,rings*2+1,1,0)), IF( (ABS((SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0)))>ABS((SEQUENCE(rings*2+1,,rings,-1)*SEQUENCE(,rings*2+1,1,0))))*((SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0))<=0), 4*(SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0))^2-(SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0))-(SEQUENCE(rings*2+1,,rings,-1)*SEQUENCE(,rings*2+1,1,0)), IF( (ABS((SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0)))<=ABS((SEQUENCE(rings*2+1,,rings,-1)*SEQUENCE(,rings*2+1,1,0))))*((SEQUENCE(rings*2+1,,rings,-1)*SEQUENCE(,rings*2+1,1,0))>0), 4*(SEQUENCE(rings*2+1,,rings,-1)*SEQUENCE(,rings*2+1,1,0))^2-(SEQUENCE(rings*2+1,,rings,-1)*SEQUENCE(,rings*2+1,1,0))-(SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0)), 4*(SEQUENCE(rings*2+1,,rings,-1)*SEQUENCE(,rings*2+1,1,0))^2-3*(SEQUENCE(rings*2+1,,rings,-1)*SEQUENCE(,rings*2+1,1,0))+(SEQUENCE(,rings*2+1,-rings,1)*SEQUENCE(rings*2+1,,1,0)) ) ) ) ) The results were mindboggling, as it was just as efficient as your lazy evaluation version (both of which are notably faster than defining each variable once in a standard LET statement). This absolutely flies in the face of conventional logic and completely contradicts LET's claim to fame. I am shocked. 😮8Views1like0CommentsRe: Cumulative Sum of Each Column or Row
Patrick2788 Good to know! Your Spiralλ function is definitely proof of concept... I tested it as written, with all calculations delayed until delivery vs having all variables evaluated as they are encountered, and it was notably faster, which is surprising to say the least, considering the sheer number of times both x() and y() are being re-evaluated (which in turn causes j() and i() to also re-evaluate). A simple test to prove delayed calculations are re-evaluated each time they are called is: =LET(f,LAMBDA(RAND()),VSTACK(f(),f())) which returns 2 different random numbers vs: =LET(f,LAMBDA(x,LAMBDA(x))(RAND()),VSTACK(f(),f())) which returns 2 identical random numbers. In the limited research I did on this topic, I read that with eager evaluation, resources are consumed up front (memory and CPU time), which can be resource-intensive if the results are not needed. This is the confusing part, because every variable in your Spiralλ function is needed and used (with the exception of d_). The underlying reasoning for the performance boost is beyond my understanding, lol. Good on you for discovering this method! ;)42Views1like2CommentsRe: Cumulative Sum of Each Column or Row
LOL, naughty, as in recycling the same variable names, a and f, in 2 locations (especially f, which defines the sub-function name, as well as one of its arguments)? Yes, this can be done, although I typically avoid it for clarity and debugging purposes. Even recursive functions defined within LET can use the same variable for the function name, as well as its combinator, e.g. =LET(fn,LAMBDA(fn,arg,... vs =LET(fn,LAMBDA(me,arg,... but I find the latter easier to read. Glad to have helped. Cheers!56Views1like0CommentsRe: Cumulative Sum of Each Column or Row
While I can't speak on the technical aspects of the concept of lazy evaluation, I do feel there is a time and place for it (and this doesn't appear to be one of those times). In a recent discussion, The Diagonal Suite: Gentle thunking goes a long way!, it was suggested that using thunks to delay eager evaluation whenever possible in generalized Lambda development will always result in quicker calculation times. However, a simple Timer test shows this to not always be the case. When testing your function as written with a large array, SEQUENCE(1000000,10), the results on my system were 3,500ms by_col and 3,000ms by_row on average (tested multiple times). I then modified the function definition by removing all 4 instances of LAMBDA(...)() and the average calculation times dropped to 2,500ms by_col and 2,000ms by_row. The formula I used to conduct the test was: =Timer(A1,LAMBDA(x,CumSum(SEQUENCE(1000000,10),x))) ...where A1 was a checkbox to toggle the by_row argument on and off. Aside from that, it's also possible to simplify the function definition by modifying the sub-function with additional parameters to swap ROWS with COLUMNS and VSTACK with HSTACK (as well as the rows and [columns] arguments of TAKE and DROP), which will eliminate having to check IF(by_row,...) twice: = LAMBDA(array,[by_row], LET( fn, LAMBDA(a,f₁,f₂,i,[j],IF(f₁(a)=1,a,a-f₂(0,DROP(TAKE(a,i,j),j,i)))), IF( by_row, fn(SCAN(0,array,SUM),ROWS,VSTACK,,-1), fn(TRANSPOSE(SCAN(0,TRANSPOSE(array),SUM)),COLUMNS,HSTACK,-1) ) ) ) Furthermore, the check for a single row or column can also be eliminated by rearranging the order of VSTACK/HSTACK and DROP (stack first to prevent DROP from erroring): =LAMBDA(array,[by_row], LET( fn, LAMBDA(a,fx,i,[j],a-DROP(fx(0,TAKE(a,i,j)),j,i)), IF( by_row, fn(SCAN(0,array,SUM),VSTACK,,-1), fn(TRANSPOSE(SCAN(0,TRANSPOSE(array),SUM)),HSTACK,-1) ) ) ) This also shaved an additional 200ms off the average calculation times in my tests. These are just my humble observations/suggestions. Your function is already very efficient as is. ;) Kind regards.19Views1like6CommentsRe: Excel
Lorenzo Testing this strategy by attaching my files first, then typing all of this. Also testing a potential workaround for macro-enabled workbooks by renaming the file with a .zip extension (to be renamed and removed after downloading). If it works, there will be no further edits to this reply (2 files attached). EDIT: the 1st attachment (.xlsx) worked, but the 2nd attachment (.xlsm.zip) was purged. Looks like a promising strategy, but still subject to other "spam" filtering rules.31Views1like1CommentRe: Excel
mathetes Many of the Excel forums that I frequent seem to be experiencing similar drops in the quantity (and quality) of questions being asked. I get the feeling that many users (especially those at an intermediate skill level) are turning to A.I. tools such as Chat GPT, Copilot, Agent Mode, etc. as their primary source of assistance nowadays. There used to be 1 or 2 questions each day that would spark my interest, but now it's more like 1 question every other week (which are typically in the form of a "challenge" shared by another contributor). I think we're on the verge of extinction, lol.61Views0likes0CommentsRe: Excel Challenge - Pivoting poorly structured data
Good stuff, Peter! Nice demonstration of the power of your custom function collection for evaluating nested arrays. I see the majority of the other dynamic array solutions offered up in the comments section of that LinkedIn post use some variation of DROP-REDUCE-VSTACK to perform each step on a row-by-row basis. In this particular situation, however, it is possible to conduct a TOCOL-broadcasting method (unpivot) over the entire table and pass the normalized fields directly to PIVOTBY: =LET( num, LEN(Categories)-LEN(SUBSTITUTE(Categories,";",))+1, arr, TEXTBEFORE(TEXTAFTER(";"&Categories&";",";",SEQUENCE(,MAX(num))),";"), fnλ, LAMBDA(err,LAMBDA(vec,TOCOL(IF(err,arr,vec),2)))(ISERROR(arr)), PIVOTBY(HSTACK(fnλ(IF(ISBLANK(Region),"Unknown",Region)),fnλ(YEAR(OrderDate))),TOCOL(arr,2),fnλ(Amount/num),SUM,0,0,,0) ) Or with CHOOSEROWS and a single broadcast of the table row IDs: =LET( num, LEN(Categories)-LEN(SUBSTITUTE(Categories,";",))+1, arr, TEXTBEFORE(TEXTAFTER(";"&Categories&";",";",SEQUENCE(,MAX(num))),";"), fnλ, LAMBDA(rId,LAMBDA(vec,CHOOSEROWS(vec,rId)))(TOCOL(IF(ISERROR(arr),arr,SEQUENCE(ROWS(arr))),2)), PIVOTBY(HSTACK(fnλ(IF(ISBLANK(Region),"Unknown",Region)),fnλ(YEAR(OrderDate))),TOCOL(arr,2),fnλ(Amount/num),SUM,0,0,,0) ) Kind regards.47Views5likes1CommentRe: Is there a way to work around Circular reference in Dynamic Array?
You're welcome. Alternatively, for this specific scenario, since the flow of the 2 dynamically calculated columns is row-major order (left-to-right, top-to-bottom), the following would also work: =SCAN(0,EXPAND(Table1[Col1],,2),LAMBDA(a,v,IF(ISNA(v),IF(a<0,10,5),IF(v<0,v+a,0)))) EXPAND is used to add a second column to Table1[Col1], which is filled with #N/A errors because the optional [pad_with] argument is not specified. This is then used to determine which formula to evaluate by testing if the current value ISNA. Cheers!55Views1like1CommentRe: Is there a way to work around Circular reference in Dynamic Array?
One way to achieve this is to utilize a CHOOSE based lambda function to store and recall multiple items at each iteration of SCAN, then MAP the results to return each individual item. For example: =LET( val, LAMBDA([a],[b], LAMBDA(x, CHOOSE(x, a, b))), num, SEQUENCE(, 2), arr, SCAN(val(), Table1[Col1], LAMBDA(acc,one, LAMBDA(two, val(two, IF(two < 0, 10, 5)))(IF(one < 0, one + acc(2), 0)))), MAP(IFNA(arr, num), IFNA(num, arr), LAMBDA(f,x, f(x))) ) Replace Table1[Col1] with the reference to your dynamic column 1 as needed.130Views3likes0CommentsRe: Calculate overlapping hours
Thanks Peter! I think practical use-cases for GROUPBY-THUNK are few and far between, but it's a viable option when the situation calls for it. I used it recently for the advent of code 2025 - day 6 challenge, so it was still fresh on my mind. Ultimately, my goal here was to create a single LAMBDA to pass to the function argument of GROUPBY but settled for the intermediate thunks step because I wasn't entirely satisfied with my first attempt. Revised version is a bit more to my liking: = GROUPBY( tblData[Supervisor], MAP(tblData[Date] + tblData[Start Time], tblData[Date] + tblData[End Time], VALS2), GroupOverlapHours, 0, 0,, tblData[Start Time] < tblData[End Time] ) Where GroupOverlapHours is defined as: = LAMBDA(vals_vector, LAMBDA(vec, SUM(BYROW(GROUPBY(-EVALS(vec, 1, 1), EVALS(vec, 2, 1), MAX, 0, 0), SUM)) * 24)( SCAN(VALS2(), SORTBY(vals_vector, EVALS(vals_vector, 1, 1)), LAMBDA(acc,val, LAMBDA(beg,end,low,upp, IF(AND(beg >= low, beg <= upp), VALS2(low, MAX(upp, end)), val))( val(1), val(2), acc(1), acc(2) ) ) ) ) ) I've been experimenting a lot recently with curried functions, hence the seemingly excessive use of nested LAMBDA's instead of LET. I couldn't have imagined writing something like this a year ago, but your discussions/teachings on thunks set me on the path of discovery. I owe you a debt of gratitude my friend! 😉7Views1like0CommentsRe: Calculate overlapping hours
Nice logic. As a LAMBDA function, it can be applied to the results of GROUPBY-THUNK to handle all supervisors: = LET( grp, GROUPBY(tblData[Supervisor], tblData[Date] + tblData[[Start Time]:[End Time]], THUNK, 0, 0,, tblData[Start Time] < tblData[End Time]), HSTACK(DROP(grp,, -2), MAP(CHOOSECOLS(grp, -2), CHOOSECOLS(grp, -1), GroupOverlapHours)) ) Where GroupOverlapHours is defined as: = LAMBDA(startTime,endTime, LET( starts, EXPAND(startTime(),, 2, 1), finish, EXPAND(endTime(),, 2, -1), stack, SORT(VSTACK(starts, finish)), headcount, SCAN(0, TAKE(stack,, -1), SUM), event, VSTACK(TAKE(stack,, 1), TAKE(stack, -1, 1)), duration, DROP(event, 1) - DROP(event, -1), SUM(IF(headcount, duration)) * 24 ) ) Alternative definition, using my VALS/EVALS functions: = LAMBDA(startTime,endTime, LET( one, startTime(), two, endTime(), arr, SCAN(VALS3(), SORTBY(MAP(one, two, VALS2), one, 1, two, 1), LAMBDA(acc,val, LAMBDA(beg,end,low,upp,lId, IF(AND(beg >= low, beg <= upp), VALS3(low, MAX(upp, end), lId), VALS3(beg, end, lId + 1)))( val(1), val(2), acc(1), acc(2), acc(3)) ) ), grp, TAKE(GROUPBY(EVALS(arr, 3, 1), arr, LAMBDA(x, INDEX(x, ROWS(x), 1)), 0, 0),, -1), SUM(EVALS(grp, 2, 1) - EVALS(grp, 1, 1)) * 24 ) ) Additional resources: https://gist.github.com/RackofLambda12Views4likes2CommentsRe: Return of the Corkscrew with Spilled Arrays
You're welcome, Peter! You may also want to change your profile settings, if you haven't already done so, to disable the default "threaded" conversation view. Under My Settings > Preferences > Replies and Comments, change Display format to Linear and Default sort order to Newest to Oldest. Sergei was kind enough to point this out to me in another thread.35Views1like1CommentRe: Excel
That's interesting. I'm using MS Edge on Windows 11, and I get a gear menu icon in the top-right corner of my screen, which allows me to edit a discussion I started... Glad to hear the edit worked for you with your reply, although it sounds like it might not be a reliable workaround 100% of the time as mentioned by SnowMan55. So disappointing to see what's become of this forum... it used to be my favorite one.122Views3likes4CommentsRe: Excel
If you edit your original post/reply immediately afterwards and reattach the exact same file, does it stick? File attachments have been removed from my two most recent replies, but editing those replies and reattaching the same files immediately thereafter has worked for me.201Views1like7CommentsRe: Need help with excel
Here's a couple additional options for MS365... Shift columns: =LET( tbl, A2:N6, rng, DROP(tbl,1), beg, INDEX(rng,,1), end, INDEX(rng,,2), val, DROP(rng,,2), low, MIN(beg), upp, MAX(end), fnλ, LAMBDA(b,e,(YEAR(e)-YEAR(b))*12+MONTH(e)-MONTH(b)), dur, fnλ(beg,end)+1, num, SEQUENCE(,fnλ(low,upp)+1,0), hdr, EOMONTH(low,num), cId, 1+num-fnλ(low,beg), rId, SEQUENCE(ROWS(rng)), arr, IF((cId>0)*(cId<=dur),INDEX(val,rId,cId),""), HSTACK(TAKE(tbl,,2),VSTACK(hdr,arr)) ) Unpivot/repivot: =LET( tbl, A2:N6, rng, DROP(tbl,1), beg, INDEX(rng,,1), end, INDEX(rng,,2), val, DROP(rng,,2), dur, (YEAR(end)-YEAR(beg))*12+MONTH(end)-MONTH(beg)+1, num, SEQUENCE(,COLUMNS(val),0), arr, IFS(num<dur,EOMONTH(+beg,num)), err, ISERROR(arr), rId, SEQUENCE(ROWS(rng)), pvt, PIVOTBY(TOCOL(IF(err,arr,rId),2),TOCOL(arr,2),TOCOL(IF(err,arr,val),2),SINGLE,0,0,,0), HSTACK(VSTACK(TAKE(tbl,1,2),INDEX(rng,TAKE(DROP(pvt,1),,1),{1,2})),DROP(pvt,,1)) ) See attached...2Views2likes0CommentsRe: Issues attaching files to posts
mathetes I just had a file attachment removed after posting a reply here: Practical use of row delimiter with textsplit Immediately thereafter, though, I edited my reply and re-attached the same file (with the same filename consisting of underscores instead of spaces) and it worked. So, the filename doesn't appear to be the primary issue after all. 😏46Views0likes2CommentsRe: Practical use of row_delimiter with TEXTSPLIT
The col_delimiter is used to spill text across columns, whereas the [row_delimiter] is used to spill text down rows. While the col_delimiter is technically a required argument, it can be omitted if the [row_delimiter] is provided. For example, =TEXTSPLIT("a,b,c",,",") will return {"a";"b";"c"} as a vertical vector. Use both delimiters to spill a 2D array of text. For example, =TEXTSPLIT("a,b;c,d",",",";") will return a 2 x 2 array consisting of {"a","b";"c","d"}. Use the optional [pad_with] argument to hide #N/A values, which result from a mismatched number of row and/or column delimiters in the text string (returning a jagged array). For example, =TEXTSPLIT("a,b;c",",",";",,,"") will return a 2 x 2 array consisting of {"a","b";"c",""}, which otherwise would have returned {"a","b";"c",#N/A} if the [pad_with] argument was not set. Knowing this, many users find the simple syntax of TEXTJOIN and TEXTSPLIT convenient and easy to understand and will use them together with either BYROW-FILTER or GROUPBY to handle certain data transformation scenarios and overcome the "nested arrays are not supported" issue. A typical example might look something like this: =LET( grp, GROUPBY(tblData[Team], tblData[Member], ARRAYTOTEXT, 0, 0), HSTACK(TAKE(grp,, 1), TEXTSPLIT(TEXTJOIN(";",, DROP(grp,, 1)), ", ", ";",,, "")) ) To be clear, there are many ways to accomplish this task, and this is not the method I would personally choose. While it's relatively short and easy to follow, it's also limited in the amount of data it can handle, due to TEXTJOIN's text limit of 32,767 characters. Having said that, it's a perfectly viable option for smaller datasets.12Views2likes0CommentsRe: Issues attaching files to posts
Good catch mathetes ! If this indeed turns out to be the source of the file attachment issue that has plagued this forum of late, it seems like something that could easily be corrected by the developers. Standard URL encoding should convert spaces in filenames to %20. This would also explain why I haven't experienced this issue yet, when so many others have, as I typically use underscores instead of spaces when naming files that will be uploaded to the web. Hopefully they fix it, though, since major problems like this appear to be driving users and contributors away from this forum.55Views0likes0Comments
Recent Blog Articles
No content to show