Obtain jupyter notebook-like output from InteractiveShell.run_cell() command

I have an object in my local environment, and I'm trying to run code originating from a jupyter notebook. The code needs to access to this object. Below is a mwe. The output of the #(current) method is a pandas dataframe, while the output from a jupyter cell will be an html view of this dataframe (see #(desired)). Is there a way to get the below output (desired) with the above method (current)?

from IPython.core.interactiveshell import InteractiveShell
import pandas as pd
import nbconvert, nbformat

#(current)
shell=InteractiveShell.instance()
shell.user_ns["importantElement"]=[12, 24, 43, 12]
output = shell.run_cell("a=pd.DataFrame({'a':importantElement})\na")
print(f"current output: {output.result}")

#(desired)
ep=nbconvert.preprocessors.ExecutePreprocessor()
ep.km=ep.kernel_manager_class()
ep.km.start_kernel()
ep.kc=ep.km.client()
ep.kc.start_channels()
ep.kc.wait_for_ready(timeout=500)

cell =nbformat.v4.new_code_cell("import pandas as pd\na=pd.DataFrame({'a':[12, 24, 43, 12]})\na")
ep.preprocess_cell(cell,"",1)
print(f"desired output: {cell.outputs}")
ep.kc.stop_channels()
ep.km.shutdown_kernel()

This is just an example. My object is more elaborate than this list. Also, it needs to work with all kinds of output (pandas dataframes, graphs, plotly graphs etc). If there is a method to share local variables with the jupyter kernel, than that will be a solution to my problem as well (no pickle, preferably no writes to the file system).