Get the needed Planner Plan Tasks from the server, rahter than get all the tasks to filter them

I have a power apps canvas application >> where the users select the AssignedTo user + select single or multiple Tasks status >> then the canvas app shows the related planner tasks, as follow:-

Currently i am implementing this as follow;-

1) Inside the Screen OnVisible property, i am looping through all groups >> all plans >> all tasks, as follow and i build the finalResult collection, which i will pass to the power apps gallery:-

Set(varInProgress,true);

ClearCollect(finalResult,{PlanTitle:"",TaskTitle:"",Status:"",AssignedToUserId:"",DueDate:"",TaskID:"",PlannerID:"",GroupID:""});
Clear(colRelatedGroupsIds);
Clear(colRelatedPlanners);
Clear(colRelatedPlannerTasks);

ClearCollect(colTeamSitesItem,Filter('Team Sites',1=1));
ClearCollect(colAllGroups,Office365Groups.ListGroups().value);
ForAll(colAllGroups As group,
ForAll(colTeamSitesItem As i2,

If(Lower(group.displayName)=Lower(i2.'Team Site Name'),
Collect(colRelatedGroupsIds,group.id))));

ForAll(colRelatedGroupsIds As group,
Collect(colRelatedPlanners,
Planner.ListGroupPlans(group.Value).value);

ForAll(colRelatedPlanners As planner,
Collect(colRelatedPlannerTasks,
Planner.ListTasksV3(planner.id,group.Value).value);
ForAll(colRelatedPlannerTasks As task,
ForAll(task._assignments As taskAssignment,
Patch(finalResult,Defaults(finalResult),
{
PlanTitle:planner.title,
AssignedToUserId:taskAssignment.userId,
Status:If(Text(task.percentComplete)="100","Completed",
   If(Text(task.percentComplete)="50","In Progress",
  If(Text(task.percentComplete)="0","Not Started"))),
TaskTitle:task.title,
DueDate:Text(DateTimeValue(task.dueDateTime), "dd/mm/yyyy"),
TaskID:task.id,
GroupID:group.Value,
PlannerID:planner.id
})


));RemoveIf(colRelatedPlannerTasks,true));RemoveIf(colRelatedPlanners,true));
Set(varInProgress,false);

2) then i am filtering the tasks inside the finalResult collection, here is my Items property for the gallery:-

Switch(SortComboBox.Selected.Value,
"Due Date",
SortByColumns(Filter(finalResult,AssignedToUserId=assignedToComboBox.Selected.Id
&&
(IsEmpty(StatusComboBox.SelectedItems)||


Status in StatusComboBox.SelectedItems.Value



)

),
"DueDate",SortOrder.Ascending

),
"Status",
SortByColumns(Filter(finalResult,AssignedToUserId=assignedToComboBox.Selected.Id
&&
(IsEmpty(StatusComboBox.SelectedItems)||


Status in StatusComboBox.SelectedItems.Value



)

),
"Status",SortOrder.Ascending

),
"Plan",
SortByColumns(Filter(finalResult,AssignedToUserId=assignedToComboBox.Selected.Id
&&
(IsEmpty(StatusComboBox.SelectedItems)||


Status in StatusComboBox.SelectedItems.Value



)

),
"PlanTitle",SortOrder.Ascending

)
)

currently this is working well, where the app will hang a bit to get all the data (till the variable varInProgress is set to false), then the filtering will happen so quickly .. but i have an issue with this appraoch as we have more groups,plans and tasks.. so how i can make my app more extensible? where i only get the tasks from the server based on the user selection, rather than getting all the tasks and filter them on the user browser.. any advice?