In time, the deficiencies stemming from the lack of processes or memory management became apparent as routers became more complex. The router OS evolved to the next stage of having an embedded OS such as QNX or VxWorks for process and memory management. From there, it was a small step to substitute the embedded OS with BSD or Linux.
Despite these changes to the underlying OS, the router remained a black box or an embedded system in its personality. Protocols were the de jure mode of communication and access. For example, SNMP was developed as a way to manage the boxes, but it ended up being used only for monitoring. Even at that it did a poor job, because MIBs had to be developed and additional code written to access the counters. Since most testing and use was via the command line, SNMP quickly devolved into a "by the way" mechanism. That it was cumbersome and slow didn't exactly add to its charm.
Configuring the boxes required screen-scraping the command-line output. The CLI itself was modal in nature. To configure a feature, you had to traipse down multiple levels: "configure terminal," "interface g1/1," "router OSPF," and so on before you got to work on the features of interest. This meant that scripting the configuration was not unlike pulling teeth. It did not help that the embedded nature of the box also meant the CLI varied quite a bit across (and many times even within) vendors. Thus, not only was screen-scraping for configuration painful, it could also change so much that developing common tools was difficult. As a consequence, the network administrator's toolkit remained woefully inadequate and antediluvian.
We then see that the reason networks are so hard to manage is rooted in the model of the OS used to run the networking gear. In other words, the solution lies in addressing the limitations of the router operating system rather than in networking per se.
Two roads diverged in the wood...
We come to a fork in the road. One road leads us to NETCONF and talk of northbound and southbound APIs and such. The other leads to Linux and its well-understood, open, mature, and vibrant ecosystem. In other words, one way is to attempt to bolt an improved management and a programmatic API to existing router OSes, and the other is to switch to an OS that already has all of those characteristics.