An introduction
Declaration of intents and scope
A bird’s eye view
Welcome to my technical and philosophical newsletter.
The intent is to elaborate and zoom, at different level of abstraction, focusing on the concepts, the processes, and when possible the nuts and bolts - yes equations, codes, diagrams, circuits, etc - which run the show of today’s products.
telecom:
wireless telecommunications, high speed landline and IP-almost-everywhere.
robotics:
dexterity in robots and also human-like dexterity robots (even though without sensory feedback), visual feedback and haptics.
quadcopters, UAVs with different levels of autonomous obstacle avoidance.
language processing:
language translators (deepL, yandex, and google translator among others).
chatbots, conversations within a wide (chatGPT, Amazon AI, IBM Watson, Google’s Bard) or restricted medical domain (google’s Med paLM and others).
medical image processing:
interpretation of MRI scan, x-ray scans, CAT scan images.
human voice and automated sound processing:
answering machines with voice-guided menu navigation.
music tunes recognition (Shazam).
These innovative products seemingly not closely related are the results of a close knit family of ideas and techniques, currently denominated as “A.I.” (as if they were non existent before..):
image processing and image recognition capabilities.
audio signal processing with voice recognition, music tunes recognition, background noise recognition.
large language models and other related applied mathematics techniques.
At least three challenges are left:
self driving cars - in which environments, and what about ethics when choosing which danger to avoid ?
natural language understanding - challenging centuries long philosophical questions of linguistics and ontology.
mobile robots exhibiting human-like autonomy in environment navigation, whether human-like or not, possibly applied to warfare - here besides technology ethical issues arise.
The very definition of the challenges is already challenging, and their ethical implications cannot be lightly dismissed.
One of the purposes of writing this newsletter is to give a perception of the nature of the last three problems above, devoid of marketing hype.
Converging Technologies
“..Cybernetics, in the 1940s, Wiener used cybernetics as an umbrella term to refer to control and communication in both the animal and the machine. In the following decades, the term has been defined in various ways by different researchers, and because of this, cybernetics has been perceived rather negatively as a “nomad science”.”. 1
Cybernetics provides a unified frame for systems employing control and systems theory, information and coding theory.
Artificial Intelligence research started in 19562 with far too optimistic expectations, many setbacks, which caused periodical withdrawal of investment: so called "A.I." winters. After each winter, after some years, a spring would come bringing a change of the dominant paradigms.
In fact, the latest breakthroughs in A.I. came from disciplines which fit squarely within Cybernetics, based on the typical theoretical background of computer, electronics, control, telecom engineering rather than what is considered strictly computer science, that is, dealing with discrete computation models.
An absolutely crucial part, in the very possibility of adopting deep neural networks, was the availability of computation power, fast access large memories, and huge mass storage memories.
The great results we see are due to the convergence of progress in solid state physics and semiconductor technologies and concurrent development of statistical signal processing, control theory and better tool chains ( we can argue that without sufficiently powerful hardware we would be still writing software only in Fortran77, and assembly languages for system level programming).
In motor control, for instance, since the ‘80’s, the availability of the IGBTs and of high performing DSPs/MCUs has been a game changer for PWM-based, Space Vector Modulation, Field Orientation Control sophisticated control strategies.
In wireless telecommunications, bandwidth increase and robustness w.r.t. radio link channel increase due to computationally expensive OFDM modulation schemes and forward error coding (Viterbi, Turbo, later LDPC) their implementation made possible, again, due to very large scale circuit integration.
A crucial role is played by proper tooling in hardware design, which evolved a great deal: in the 4-bit processor era, processors were designed at transistor level, not even gate level. Nowadays, for FPGAs, if a non-optimized, but reasonably performing design is needed, one can resort to High Level Synthesis, which still needs awareness of digital hardware for decent results, yet handles most of the burden; the days in which digital circuits were drawn at gate level, like Federico Faggin did for the Z80, are gone.
Machine learning techniques do not come from nowhere, all of a sudden, they are the latest, sophisticated applications of concepts developed from Wiener’s discrete adaptive filter, matrix theory (as old as late 1800’s), mathematical optimization, statistical methods, scaled up and improved to handle numerical issues. And they would be non-implementable without the progress made in several technologies.
Irrational optimism, once again
Namely due to the progress achieved, and exactly as it happened at the very beginning of A.I. research, an irrational optimism is predominant in the quest of autonomous driving and language comprehension, and in a larger scope, having robots approaching human capabilities in environment navigation.
Usually applied mathematics and logic lead over computational capabilities, there are issues however, different than speed of computation and data storage and latency which are neglected, and they can be real showstoppers.
Why subscribing ?
If you survived up to this point, and perhaps you enjoyed reading, maybe you could think about subscribing.
Most content is going to be free of charge, yet a subscription to support my work would be welcome.
As in “Cybernetics where shall we go?”, Qiangfu Zhao, John Brine and Dimitar Filev, 2013 IEEE International Conference on Cybernetics.
McCarthy, J., Minsky, M., Rochester, N., Shannon, C.E., A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence., http://raysolomonoff.com/dartmouth/boxa/dart564props.pdf