Big Data

Big data problems are often considered difficult to solve using traditional methods due to the lack of scalability of the queries and storage methods. At Ion Machines, we have experience in developing innovative caching schemes and query methods that reduce the computational overhead and allow systems to scale. Additionally, we have experience implementing SSD complemented storage solutions. Such systems allow you to add intelligence to the allocation of resources and can be designed modularly so as to add high speed SSD only at the rate your system needs it and budget can afford it.

Beginning as an outgrowth of our defense applications, we at Ion Machines quickly learned that some systems have large data streams that must be pre-processed in real time to reduce the storage burden. From this work, we realized that there was a broader application of the algorithms and expertise we have developed in the commercial sector.

For instance, many existing systems shard their data without taking into account the typical queries that are applied to the data. At Ion Machines, we’ve developed algorithms that make use of correlations that typical queries test so as to shard the data by a “synthetic predicate”. This avoids the typical communication delays that occur as data is moved between shards in order to compute overall coorelations.

Parallel processing is another area that we have developed an expertise. Many algorithms claim to operate in parallel, but a typical profile often reveals processors misused waiting for data.

Learn More

Mobile
Machine Learning
Back