AssuredTest(tm) - Incremental Testing in CI/CD Lower Stages
I have introduced a new pair of products, AssuredTest(tm) and AssuredTest-C(tm). They are both ML driven but by functor models, not stochastic. This is an https://github.com/AutonomicAI/functor-resnet on GitHub for anyone interested. There are some various features with these products that my patent attorney advised me to only disclose under NDA. Knowing the features would be very easy to develop indepently for any competitor in incremental testing. The AssuredTest-C is a highly optimized companion to AssuredCode-C which generates bare metal MISRA C code in a benchmarked mean of 0.36 ms and I have created "https://www.functormodel.ai/assuredprocess" which uses blue-green digital twins, and I can support safe end-end generate to deploy in under 10 ms. This is not in competition with LLMs as AssuredCode-C is spec-based and has no NLP built in. AssuredTest will not run a test when there has been no code, dependency, configuration, or environment change that could affect its result, much like an incremental compiler. It maintains an internal registry of test results and their relevant dependencies. When nothing affecting a test has changed, AssuredTest returns the most recent valid result rather than unnecessarily executing the test again. This can significantly reduce the compute and energy consumed by large regression test suites. Some other features guarantee 100% code line coverage. Your team can avoid writing so many unit tests and focus on real business cases. A Cucumber/Gherkhin interface is used for "look ahead" style testing which may need to happen if the code is being generated. Through the years working with CI/CD I did always feel it was a design smell to have the same test executed so many times when the outcome was unquestionably known. I will say upfront as well, I am environmentally minded so I like to "invent green". I asked GPT how much energy would be saved if every IT shop in America used this instead of rerunning unit test harnesses in their dev & test environments? It estimated a potential 5 TWh. Of course that's a guesstimate in many ways, depending on many factors and of course advanced test environments like UAT and production are not included. I would obviously not advise anything other than running a certified test harness for production systems.Functor Model Transparency and Auditability
https://github.com/AutonomicAI/functor-resnet/tree/main/slm_template shows a refactoring of the model into separate files. For example, a file f.py is used for the model function, delta.py for model changes that are applied and w.py for the weights function. Every changed to the model is logged to an enterprise streaming event log and versions in a source code repository for these files can be referenced. The model.py code now references these other files as well as containing model functionality. It resembles an orchestrator or a controller in a sense. model.py => { load current functions, route inference, invoke components, invoke state/context representation functions, apply governed updates, emit provenance } With a specialized VCS tied into normal engineering workflow, a model change can become an ordinary governed change record. Then an auditor can traverse the chain in either direction answering: who changed it? what changed? why it changed? which requirement authorized it? what evidence supported it? which tests passed? which model version resulted? That is much richer than conventional model observability. So, there are really two layers: prediction audit - explain why a certain output prediction was given evolution audit - explain the current manifestation of the model Simple integration of the event log with the version control system and the project management tooling such as Microsoft Planner Functor Models may be unusually strong on the second because the change itself is a first-class artifact. The Microsoft Planner/Jira/PR integration matters because we are not asking enterprises to invent a new governance process. We are mapping model evolution into processes they already use for software change control. The specialized version control is a separate topic in as sense as we could see some significant energy savings via reuse in learning/training computation. A VCS with sophisticated semantic searching, fragmentation matching (we computed learning with a expression containing reusable piece for example) could save computation. This is preliminary research and requires further investigation of course. "Treat a learned model change with the same rigor as a production code change". For regulated environments, that is a very compelling proposition.