Backend.AI component composition diagram
Dear AI Network Community,
Hello, this is Jun-ki Kim from AI Network. I am leading the design and development of Backend.AI framework based on my experience in developing GPU-accelerated distributed processing systems. Let me tell you more about Backend.AI, which was briefly mentioned in the last post. It may be a little difficult for non-expert to understand AI and AI development-related terms, but I’ve written them down as easily as possible, so I’d appreciate your effort to read them.
Do you know a movie about sushi master, “Jiro Dreams of Sushi”? In order for a master like Jiro to pick up the sushi in front of the guests, it takes invisible sweat and effort from an assistant chef and a junior apprentice in the kitchen. It’s backed up by the hassle but necessary work such as cooking rice properly, selecting the best ingredients in the market every morning, cleaning and aging the fish, and so on. Similarly, to promote the birth of good artificial intelligence, someone needs to do a lot of tedious pre-work for the developer.
Back to tech talk, a lot of deep learning frameworks — TensorFlow, PyTorch, Caffe, CNTK, etc. — have been developed in the last few years, and a lot of cloud services like AWS, Azure, GCP and others comes out there to host those frameworks. However, it is still difficult to optimize it from a cost perspective. For example, to reduce the cost of virtual machines, you need to utilize the spot rate system and pay attention to when the computation begins and ends so that you can use the exact amount of computing resources required. And simultaneously you should keep up with up-to-date version of the Machine Learning framework, and need to manage the virtual machine image to run the existing code. It’s difficult for machine learning model developers to care about all of these management tasks.
Our development team introduced Backend.AI to alleviate these infrastructure concerns from AI developers. Backend.AI is a tool focused on the orchestration of computing resources between Deep Learning framework and cloud infrastructure. It returns the user code and data entered through the API in high density within given computing resources by utilizing the container technology, minimizing mutual interference.
Specifically, it enables developers to split high-performance GPUs into smaller pieces without the need for a separate HW virtualization solution. It also supports resource sharing and constraints for legacy calculation libraries that traditional container solutions such as Docker do not cover.
Backend.AI provides auto scaling in cloud environments and policy-based scheduling in cluster environments. Now, developers have an environment where they can focus on core development projects like Jiro.
As financial institutions and others are increasingly adopting cloud, computing infrastructures are moving to the cloud. As an example, Hana Bank recently decided to introduce its own public cloud service.
Given the nature of the cloud seeking benefits with economies of scale, large enterprises such as Google and Amazon have competitive advantage. But in order to “ get better “ from these clouds, you need to get used to the APIs or console interfaces that each cloud provider supports, which is too complex for even existing system engineers. Enterprise authority management systems, for example IAM from AWS, are simply obstacles for frontline engineers.
Backend.AI provides an easier user computing experience without being dependent on specific cloud providers. Especially, if you install plug-in on tools that artificial intelligence developers use most often, such as jupiter notebook, Visual Studio Code, and Atom Editor, you can edit the code and simultaneously you can try running code directly from the Backend.AI cloud or cluster. Each has an API key that allows administrators or developers to control resource usage, and can link all plug-ins to a customized command-line tool.
Additionally, Backend.AI can be installed directly in clusters owned by users in addition to the cloud. For businesses or organizations dealing with sensitive data that haven’t yet moved the data to the cloud due to private data restrictions, With Backend.AI, these entities can make use of the convenience of a commercial cloud.
Backend.AI solution already constructed appropriate technical features for use in the AI network, a blockchain network for distributed processing. Backend.AI’s ground solution provides Secure Runtime Environment (SRE) for each node of AI Network to ensure security for arbitrary code execution. And it distributes the computational resources of nodes to the node owners only as much as they want and lets them monitor that this is complied. Backend.AI’s cloud solution is a bridge that connects massive, high-performance clouds, such as Google and Amazon, to the computing infrastructure supporting the AI network, based on the advantages the Ground solution offers.
Based on these technical features, the Backend.AI development team itself participates as one or multiple admin nodes of the AI Network. Ground provides experiences and examples of AI Network’s individual node operations, and Cloud offers existing clouds as worker nodes with strong computational power to the network.
The non-dependent characteristics of the Backend.AI solution to cloud provider or underlying hardware makes it easy for node owners of various environments and configurations to participate in AIN.
This AI Network participation by the development team will serve as a reference and guide for network participants as the network grows. Also, because Backend.AI reduces the complexity of the node operation process and ensures transparency of critical transactions in the network through the blockchain protocol, it is more sophisticated for other participants.
AI Network aims to provide an open market for “computing” for AI problems based on the blockchain. This combination of Backend.AI and the blockchain has the following advantages over traditional blockchains for sharing computing resources or for solving AI problems:
Backend.AI is an AI solution that has already been developed and commercialized. This means that the development team is already delivering value to our customers from years of experience in the area of computational resource orchestration between Deep Learning framework and cloud infrastructure, container technology for user code and data, auto scaling in cloud environments, and scheduling in cluster environments. (Note that Backend.AI already supports 13 programming languages, such as Python, Julia and R, and also supports most ML toolkits, such as TensorFlow, PyTorch and Caff.) As a result, the AI network can immediately provide a service level of commercialization without going through a lengthy experiment.
As mentioned earlier, Backend.AI takes the form of a multi-platform solution that can be accessed with the same interface for cloud computing resources and directly managed clusters. Using these technologies, AI Network can start without dependence on a specific platform, which it expects will contribute to a stable supply of computing resources over the network. This also links to the price stability of the services clients receive through AI Network.
Backend.AI is basically the first node contributor to AI Network. Participants who want to make money by providing computing power or who want to solve artificial intelligence problems by using AIN can participate in job schema provided by Backend.AI to achieve the desired results in the early days of the network. This makes Backend.AI the first reference and guide to the growth of the network. Late participants in the network can develop, experiment, and post their solutions to AI Network more easily by referring to proven Backend.AI.
Thank you for reading the long article.
In the next post, CEO Kim Min-hyun, who designed the entire AI Network blockchain protocol, will introduce you in detail about the mainnet of AI Network.
Homepage: http://ainetwork.ai/
Medium(English): https://medium.com/ai-network
Medium(Korean): https://medium.com/ai-networkkr
Telegram (English): https://t.me/ainetwork_en
Email: [email protected]
Twitter : https://twitter.com/AINetwork1
Facebook: https://www.facebook.com/AINETWORK0/
Brunch : https://brunch.co.kr/@ainetwork
github : https://github.com/lablup/backend.ai
