spark-connect is a Ruby client for Apache Spark Connect, the gRPC-based decoupled client-server protocol for Apache Spark. It provides a DataFrame API closely modeled on PySpark, including SQL, relational operators, column expressions, a comprehensive functions library, typed schemas, and Apache Arrow-based result decoding.
Required Ruby Version
>= 3.1.0
Authors
Hyukjin Kwon