Home

How to feed back the results from a batch?

%3CLINGO-SUB%20id%3D%22lingo-sub-857991%22%20slang%3D%22en-US%22%3EHow%20to%20feed%20back%20the%20results%20from%20a%20batch%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-857991%22%20slang%3D%22en-US%22%3E%3CP%3EHi!%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20have%20tried%20contacting%20Azure%20technicians%20and%20he%20said%20maybe%20the%20paid%20support%20can%20answer%20me%20but%20they%20don't%20know%20the%20answer.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EMy%20project%20is%20to%20scan%201GB%20of%20unstructured%20text%2C%20quickly%2C%20by%20splitting%20it%20onto%20~128%20vCPUs%2C%20checking%20each%2010MB%20for%20word%20matches%2C%20and%20then%20feed%20back%20all%20results%20to%20one%20location%20to%20weigh-in%20and%20tally%20up%20the%20match%20counts.%20The%20question%20is%2C%20is%20there%20a%20bottleneck%20delay%3F%20If%20so%2C%20how%20long%3F%20I%20would%20need%20all%20cores%20to%20send%20the%20results%20back%20and%20meet%20up%20within%20~some%20few%20seconds.%20Further%2C%20what%20if%20the%20128%20vCPUs%20can't%20even%20send%20back%20results%20to%201%20location%2C%20or%20they%20can%20but%20they%20share%20the%20same%20bus%2FRAM%20and%20hence%20the%20individuual%2010MB%20checks%20are%20like%20500MB%20checks%20and%20basically%20no%20faster.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-857991%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EMedia%20Services%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Highlighted
Steven660
Occasional Visitor

Hi!

 

I have tried contacting Azure technicians and he said maybe the paid support can answer me but they don't know the answer.

 

My project is to scan 1GB of unstructured text, quickly, by splitting it onto ~128 vCPUs, checking each 10MB for word matches, and then feed back all results to one location to weigh-in and tally up the match counts. The question is, is there a bottleneck delay? If so, how long? I would need all cores to send the results back and meet up within ~some few seconds. Further, what if the 128 vCPUs can't even send back results to 1 location, or they can but they share the same bus/RAM and hence the individuual 10MB checks are like 500MB checks and basically no faster.