Math Functions: Absolute, Ceil, Floor, Exponential & Power Values
Last updated on: 2025-05-30
Continuing our discussion from the previous article on mathematical functions, let’s now explore some advanced mathematical operations available in Spark that can be applied to DataFrames.
In this article, we'll cover the following functions:
- abs()
- ceil()
- floor()
- exp()
- pow()
- sqrt()
Sample Dataset
Imagine you are analyzing data from a neighborhood with 5 parks. The data available includes:
+-------+--------+---------+---------+-----+
|Park ID|Distance|Perimeter|Elevation| Area|
+-------+--------+---------+---------+-----+
| 1| 1250| 150.7| 4.5|5.972|
| 2| 670| 78.3| -3.2|0.642|
| 3| 1580| 225.1| 6.0| 86.8|
| 4| 340| 108.9| -1.5|0.013|
| 5| 1100| 45.6| 5.5|1.898|
+-------+--------+---------+---------+-----+
Let us apply the above mathematical functions on this dataframe
abs() – Absolute Value
To get the absolute values of the Elevation column, we use abs() function
val absElevation = data.withColumn("Absolute Elevation",
abs(col("Elevation"))
)
absElevation.show()
Output
+-------+--------+---------+---------+-----+------------------+
|Park ID|Distance|Perimeter|Elevation| Area|Absolute Elevation|
+-------+--------+---------+---------+-----+------------------+
| 1| 1250| 150.7| 4.5|5.972| 4.5|
| 2| 670| 78.3| -3.2|0.642| 3.2|
| 3| 1580| 225.1| 6.0| 8.68| 6.0|
| 4| 340| 108.9| -1.5|0.013| 1.5|
| 5| 1100| 45.6| 5.5|1.898| 5.5|
+-------+--------+---------+---------+-----+------------------+
ceil() – Ceiling Value
ceil() is used to round up the values to the next integer.
ex. To find the ciel values of the Area of Park -
val ceilArea = data.withColumn("Ceil Area value",
ceil(col("Area"))
)
ceilArea.show()
Output
+-------+--------+---------+---------+-----+---------------+
|Park ID|Distance|Perimeter|Elevation| Area|Ceil Area value|
+-------+--------+---------+---------+-----+---------------+
| 1| 1250| 150.7| 4.5|5.972| 6|
| 2| 670| 78.3| -3.2|0.642| 1|
| 3| 1580| 225.1| 6.0| 8.68| 9|
| 4| 340| 108.9| -1.5|0.013| 1|
| 5| 1100| 45.6| 5.5|1.898| 2|
+-------+--------+---------+---------+-----+---------------+
floor() – Floor Value
floor() is used to round down the parameter values to the previous integer. Assume, we want to find the floor value of the Perimeter -
val floorValue = data.withColumn("Floor Value",
floor(col("Perimeter"))
)
floorValue.show()
Output
+-------+--------+---------+---------+-----+-----------+
|Park ID|Distance|Perimeter|Elevation| Area|Floor Value|
+-------+--------+---------+---------+-----+-----------+
| 1| 1250| 150.7| 4.5|5.972| 150|
| 2| 670| 78.3| -3.2|0.642| 78|
| 3| 1580| 225.1| 6.0| 8.68| 225|
| 4| 340| 108.9| -1.5|0.013| 108|
| 5| 1100| 45.6| 5.5|1.898| 45|
+-------+--------+---------+---------+-----+-----------+
exp() – Exponential Value
To calculate the exponential of the Area values (e^Area)
val expValue = data.withColumn("Exponential Value",
exp(col("Area"))
)
expValue.show()
Output
+-------+--------+---------+---------+-----+------------------+
|Park ID|Distance|Perimeter|Elevation| Area| Exponential Value|
+-------+--------+---------+---------+-----+------------------+
| 1| 1250| 150.7| 4.5|5.972|392.28946562499834|
| 2| 670| 78.3| -3.2|0.642|1.9002776365552259|
| 3| 1580| 225.1| 6.0| 8.68| 5884.046591336165|
| 4| 340| 108.9| -1.5|0.013|1.0130848673598092|
| 5| 1100| 45.6| 5.5|1.898| 6.672536016273524|
+-------+--------+---------+---------+-----+------------------+
pow() – Power Function
The power of a number refers to the result obtained when that number is multiplied by itself a certain number of times. Here there are two components, base and exponent. Base is the number to be multiplied and exponent is the number of times base is multiplied by itself.
ex. To raise the Perimeter values to the power of 2:
val powValue = data.withColumn("Power Value",
pow(col("Perimeter"),2)
)
powValue.show()
Output
+-------+--------+---------+---------+-----+------------------+
|Park ID|Distance|Perimeter|Elevation| Area| Power Value|
+-------+--------+---------+---------+-----+------------------+
| 1| 1250| 150.7| 4.5|5.972|22710.489999999998|
| 2| 670| 78.3| -3.2|0.642| 6130.889999999999|
| 3| 1580| 225.1| 6.0| 8.68|50670.009999999995|
| 4| 340| 108.9| -1.5|0.013|11859.210000000001|
| 5| 1100| 45.6| 5.5|1.898| 2079.36|
+-------+--------+---------+---------+-----+------------------+
sqrt() – Square Root
To find the square root of a given value, we use the sqrt(value) operator.
val sqrtValue=data.withColumn("Square Root",
sqrt(col("Distance"))
)
sqrtValue.show()
Output
+-------+--------+---------+---------+-----+------------------+
|Park ID|Distance|Perimeter|Elevation| Area| Square Root|
+-------+--------+---------+---------+-----+------------------+
| 1| 1250| 150.7| 4.5|5.972| 35.35533905932738|
| 2| 670| 78.3| -3.2|0.642| 25.88435821108957|
| 3| 1580| 225.1| 6.0| 8.68| 39.74921382870358|
| 4| 340| 108.9| -1.5|0.013|18.439088914585774|
| 5| 1100| 45.6| 5.5|1.898| 33.166247903554|
+-------+--------+---------+---------+-----+------------------+
Mathematical operations are the foundation for aggregate functions. A detailed explanation of various aggregate functions has been given in
Summary
In this article, we covered:
-
Absolute values with abs()
-
Rounding operations with ceil() and floor()
-
Exponential values with exp()
-
Power calculations with pow()
-
Square root extraction using sqrt()
-
Real-life applications using park data
These functions form a strong base for more advanced operations like aggregate functions, which are covered in our next article: Aggregate Functions