Monolith is the codename for a version of AI2 with improved extensibility. Most command shells have the concept of programs, which have well-defined input/outputs but which also implement additional UIs. E.g., a text editor implements its own UI independent of the shell.

The key idea in Monolith is to have the shell provide all program UIs, and that programs should only ever return data—data annotated with enough semantics to allow the shell to provide editing services.

For example, rather than having a program to edit a text file, the program would output some data with a few annotations:

  • The data is free-form, line-oriented text (can be edited by a line editor)
  • We provide methods (or descriptions) for updating the data when finished editing.

Core Concepts

  • A file is an Aeon file stored in AI2.files. All files have a file type (which is encoded as an extension). The file path always starts with a username. We have secondary index by username, file type, and file path. And we also have a secondary index by username, tag, and file path.
  • A program is a file of type .program, containing code stored under a user's file storage. The unique name of the program is the full filepath (which includes the username).
  • To share a file the user creates a shared space and then adds references to one or more files. Shared spaces show up as top-level objects to all users with access. We manage shared spaces in AI2.shared (or something).

Settings

We need a place to store user settings. A setting is a unit of data that is needed by one or more programs. Some settings are visible to the user, but others are not.

Storage

We need to be able to store programs.

Some ideas:

  • We keep a table of files called AI2.files. When a file is created, we assign it a unique ID. Secondary views sort by user, by user and file type, and by user and tag.
  • We store owner (username), file type, and tags in the descriptor. We also store pointers to folders that contain the file. Folders are stored as a first-class object and can be shared and nested.
  • At create time we must specify a file type from a list of well-known types.
  • AI2 provides a UI for editing text files.
  • NOTE: At some point we may need to shard the list of files. At that point we can use a hash of the username as a prefix for the file ID.

Questions

  • Where do we store program settings (non-user data)?
  • How do we share programs?
  • How do permissions work? Perhaps programs have permission to modify a set of tags?
  • Do we separate files by type (e.g., an image/photo store) or do we have a generic file system?
  • We should support the concept of tags. Perhaps file type is a special tag?
  • How do we identify programs uniquely?
  • How do we deal with program versions [particularly for shared programs]?
  • How would a program implement/use a database? Is this part of the storage system or should it be separate?
  • How do we interface with external storage (e.g., Google Drive)?

Execution

The execution engine runs programs on the server and provides control and I/O. We provide library functions for the program to accept input and generate output. The AI2 web interface coordinates to supply input and accept output.

Some ideas:

  • Each execution program has an input queue and an output queue. The AI2 interfaces does an RPC call on every user input (to post into the input queue) and gets back the current contents of the output queue. In addition, we poll every second to get any additional output.
  • We keep an array of running processes (CHexeProcess), indexed by ID.
  • Each process is in one of the following states: running, ready to run (timesliced out), waiting for an RPC result, waiting for input.
  • Each thread in the thread pool pulls a process that needs to run and runs it until it stops or until the timeslice expires.
  • When an input message comes in, we add it to the input queue and run the process until there is output, or until it stops, or until a timeslice expires.
  • The client will also periodically poll for output for a process.
  • How do we implement multi-user programs? Do we have multiple I/O queues?

Status

  • 2018-12-30: Added AI2.files and AI2.api+createFile (though not yet tested). We need the following:
    • API to list the set of files (for a user)
    • API to get a file by ID
    • API to upload a new version of the file by ID
    • Command on AI2 to create a new empty program file.
    • Command on AI2 to return a list of files, with links to open up the file.
    • AI2 page to view a file, with a button to edit.
    • AI2 page to edit a file.
    • Command on AI2 to run a file as a program.