Documentation

⌘K
Getting started with the Kensu Community Edition
Marketing campaign
Financial data report
Getting credentials
Recipe: Observe Your First Pipeline
Integration
Collectors
Connectors
Agents: getting started
Python
PySpark
Scala Spark
Databricks Notebook
Agent Listing
Docs powered by
Archbee
Agents: getting started
Databricks Notebook

Running your first Databricks Kensu program

4min

👨‍💻 Modify Notebook to use Kensu

1️⃣ Init Kensu in your code

Add a new code cell at the beginning of the Notebook

Scala
|
%scala
io.kensu.sparkcollector.environments.DatabricksCollector.track(spark, "/dbfs/FileStore/conf/conf.ini" )


ℹ️ Customizing configuration for single notebook



It is possible to overwrite the settings of the conf.ini file only for this particular notebook (i.e. conf.ini could be shared globally, and customizations applied for certain notebook when needed) with the following code:



io.kensu.sparkcollector.environments.DatabricksCollector.track(spark, "/dbfs/FileStore/conf/conf.ini", "report_to_file" -> "True")

2️⃣ Run the application

Execute the already existing program. You can find a test notebook here

3️⃣ Check the result

Now you can view the data in the Kensu App.

Initialization with Python code

In case Scala language is not convenient, it is possible to use full Python code

Python
|
def ref_scala_object(jvm, object_name):
    class_name_parts = (object_name + "$.MODULE$").split(".")
    o = jvm
    for p in class_name_parts:
      o = getattr(o, p)
    return o
kensuInit = ref_scala_object(spark.sparkContext._jvm, "io.kensu.sparkcollector.environments.DatabricksCollector")
t2 = spark.sparkContext._jvm.scala.Tuple2
properties = spark.sparkContext._jvm.scala.collection.mutable.HashSet()
#Optional: properties definition
properties.add(t2("offline_file_name", "demo_notebook_test.jsonl"))
properties.add(t2("report_to_file", "True"))
kensuInit.track(spark._jsparkSession, "/dbfs/FileStore/Kensu/conf-1.ini", properties.toSeq())




Updated 03 Mar 2023
Did this page help you?
PREVIOUS
Configure the Agent
NEXT
Agent Listing
Docs powered by
Archbee
TABLE OF CONTENTS
👨‍💻 Modify Notebook to use Kensu
Initialization with Python code
Docs powered by
Archbee