Programming language for all of AI

2023-05-02 11:45:40

Mojo combines the usability of Python with the efficiency of C, unlocking unparalleled programmability of AI {hardware} and extensibility of AI fashions.

def softmax(lst):
  norm = np.exp(lst - np.max(lst))
  return norm / norm.sum()

def softmax(lst):
  norm = np.exp(lst - np.max(lst))
  return norm / norm.sum()

struct NDArray:
  def max(self) -> NDArray:
    return self.pmap(SIMD.max)

struct SIMD[type: DType, width: Int]:
  def max(self, rhs: Self) -> Self:
    return (self >= rhs).choose(self, rhs)

Usability & Programmability

Write Python or scale all the best way right down to the metallic. Program the multitude of low-level AI {hardware}. No C++ or CUDA required.

Take a tour of Mojo

FILE_NAME.????

def type(v: ArraySlice[Int]):
  for i in vary(len(v)):
    for j in vary(len(v) - i - 1):
      if v[j] > v[j + 1]:
        swap(v[j], v[j + 1])

FILE_NAME.????

struct MyPair:
  var first: Int
  var second: F32
  
  def __init__(self, first: Int, second: F32):
    self.first = first
    self.second = second

FILE_NAME.????

def reorder_and_process(owned x: HugeArray):
  type(x)	# Replace in place
  
  give_away(x^)	# Switch possession
  
  print(x[0])	# Error: ‘x’ moved away!

FILE_NAME.????

def exp[dt: DType, elts: Int]
    (x: SIMD[dt, elts]) -> SIMD[dt, elts]:
  x = clamp(x, -88.3762626647, 88.37626266)
  okay = flooring(x * INV_LN2 + 0.5)
  r = okay * NEG_LN2 + x
  return ldexp(_exp_taylor(r), okay)

FILE_NAME.????

def exp_buffer[dt: DType](information: ArraySlice[dt]):

  # Seek for the most effective vector size
  alias vector_len = autotune(1, 4, 8, 16, 32)
  
  # Use it because the vectorization size
  vectorize[exp[dt, vector_len]](information)

Progressive Sorts

Leverage sorts for higher efficiency and error checking.

Zero Value Abstractions

Take management of storage by inline-allocating values into constructions.

Possession + borrow checker

Benefit from reminiscence security with out the tough edges.

Moveable parametric algorithms

Leverage compile-time meta-programming to write down hardware-agnostic algorithms and scale back boilerplate.

LANGUAGE INTEGRATED Auto-tuning

Mechanically discover the most effective values in your parameters to benefit from goal {hardware}.

The complete energy of MLIR

Parallel heterogenous runtime

Quick compile occasions

FILE_NAME.????

def type(v: ArraySlice[Int]):
  for i in vary(len(v)):
    for j in vary(len(v) - i - 1):
      if v[j] > v[j + 1]:
        swap(v[j], v[j + 1])

Options embody:

Progressive Sorts

Leverage sorts for higher efficiency and error checking.

FILE_NAME.????

struct MyPair:
  var first: Int
  var second: F32
  
  def __init__(self, first: Int, second: F32):
    self.first = first
    self.second = second

Options embody:

Zero Value Abstractions

Take management of storage by inline-allocating values into constructions.

FILE_NAME.????

def reorder_and_process(owned x: HugeArray):
  type(x)	# Replace in place
  
  give_away(x^)	# Switch possession
  
  print(x[0])	# Error: ‘x’ moved away!

Options embody:

Possession + borrow checker

Benefit from reminiscence security with out the tough edges.

FILE_NAME.????

def exp[dt: DType, elts: Int]
    (x: SIMD[dt, elts]) -> SIMD[dt, elts]:
  x = clamp(x, -88.3762626647, 88.37626266)
  okay = flooring(x * INV_LN2 + 0.5)
  r = okay * NEG_LN2 + x
  return ldexp(_exp_taylor(r), okay)

Options embody:

Moveable parametric algorithms

Leverage compile-time meta-programming to write down hardware-agnostic algorithms and scale back boilerplate.

FILE_NAME.????

def exp_buffer[dt: DType](information: ArraySlice[dt]):

  # Seek for the most effective vector size
  alias vector_len = autotune(1, 4, 8, 16, 32)
  
  # Use it because the vectorization size
  vectorize[exp[dt, vector_len]](information)

Options embody:

LANGUAGE INTEGRATED Auto-tuning

Mechanically discover the most effective values in your parameters to benefit from goal {hardware}.

Make the most of the complete energy of the {hardware}, together with a number of cores, vector models, and unique accelerator models, with the world’s most superior compiler and heterogenous runtime. Obtain efficiency on par with C++ and CUDA with out the complexity.

Parallelization

Mojo leverages MLIR, which permits Mojo builders to benefit from vectors, threads, and AI {hardware} models.

PYTHON

Single-threaded execution

Mojo ????

Parallel processing throughout a number of cores

Occasion

AWS r7iz.metal-16xl

Intel Xeon

Expertise true interoperability with the Python ecosystem. Seamlessly intermix arbitrary libraries like Numpy and Matplotlib and your customized code with Mojo.

Read the programming manual

def make_plot(m: Matrix):
  plt = Python.import_module("matplotlib.pyplot")
  fig = plt.determine(1, [10, 10 * yn // xn], 64)
  ax = fig.add_axes([0.0, 0.0, 1.0, 1.0], False, 1)
  plt.imshow(picture)
  plt.present()

make_plot(compute_mandelbrot())

Simply prolong your fashions with pre and post-processing operations, or change operations with customized ones. Benefit from kernel fusion, graph rewrites, form capabilities, and extra.

Mannequin extensibility

Mojo can improve the prevailing operations in your mannequin.

Enter layer

Hidden layers

Output layer

Mojo remains to be a piece in progress, but it surely’s obtainable to strive as we speak in our JupyterHub-based Playground. Run by tutorials and write your individual Mojo code.