Kensu Documentation

⌘K
Getting started with the Kensu Community Edition
Marketing campaign
Financial data report
Getting credentials
Recipe: Observe Your First Pipeline
Agents: getting started
Python
PySpark
Scala Spark
Databricks Notebook
Agent Listing
Docs powered by archbee 
4min

Running your first Databricks Kensu program

👨‍💻 Modify Notebook to use Kensu

1️⃣ Init Kensu in your code

Add a new code cell at the beginning of the Notebook

Scala
|
%scala
io.kensu.sparkcollector.environments.DatabricksCollector.track(spark, "/dbfs/FileStore/conf/conf.ini" )


ℹ️ Customizing configuration for single notebook



It is possible to overwrite the settings of the conf.ini file only for this particular notebook (i.e. conf.ini could be shared globally, and customizations applied for certain notebook when needed) with the following code:



io.kensu.sparkcollector.environments.DatabricksCollector.track(spark, "/dbfs/FileStore/conf/conf.ini", "report_to_file" -> "True")

2️⃣ Run the application

Execute the already existing program. You can find a test notebook here

3️⃣ Check the result

Now you can view the data in the Kensu App.

Initialization with Python code

In case Scala language is not convenient, it is possible to use full Python code

Python
|
def ref_scala_object(jvm, object_name):
    class_name_parts = (object_name + "$.MODULE$").split(".")
    o = jvm
    for p in class_name_parts:
      o = getattr(o, p)
    return o
kensuInit = ref_scala_object(spark.sparkContext._jvm, "io.kensu.sparkcollector.environments.DatabricksCollector")
t2 = spark.sparkContext._jvm.scala.Tuple2
properties = spark.sparkContext._jvm.scala.collection.mutable.HashSet()
#Optional: properties definition
properties.add(t2("offline_file_name", "demo_notebook_test.jsonl"))
properties.add(t2("report_to_file", "True"))
kensuInit.track(spark._jsparkSession, "/dbfs/FileStore/Kensu/conf-1.ini", properties.toSeq())




Updated 18 Jan 2023
Did this page help you?
Yes
No
UP NEXT
Agent Listing
Docs powered by archbee 
TABLE OF CONTENTS
👨‍💻 Modify Notebook to use Kensu
Initialization with Python code