Use to_string() to Cease Python from Hiding the Physique of the Printed DataFrames | by Yufeng

3-Minutes Pandas

What ought to we do to see your entire printed dataframe after the execution of a Python script?

Typically operating by means of a Python script with out reporting any errors is just not the one activity of the debugging course of. We’d like to verify the features are executed as anticipated. It’s a typical step within the exploratory knowledge evaluation to test how the information seems like earlier than and after some particular knowledge processing.

So, we have to print out some knowledge frames or important variables throughout the execution of the script, so as to test whether or not they’re “right”. Nevertheless, easy print command can solely present the highest and backside rows of the information body typically (as proven within the instance beneath), which makes the checking process unnecessarily laborious.

Normally, the information frames are within the format of pandas.DataFrame, and should you use the print command instantly, you may get one thing like this,

import pandas as pd
import numpy as np
knowledge = np.random.randn(5000, 5)
df = pd.DataFrame(knowledge, columns=['A', 'B', 'C', 'D', 'E'])
print(df.head(100))

You will have already seen that the center a part of the information body is hidden by three dots. What if we actually must test what the highest 100 rows are? For instance, we need to test the results of a selected step in the midst of a big Python script, so as to be sure that the features are executed as anticipated.

set_option()

Some of the simple options is to edit the default variety of rows that Pandas present,

pd.set_option('show.max_rows', 500)
print(df.head(100))

the place set_option is a technique that means that you can management the habits of Pandas features, which incorporates setting the utmost variety of rows or columns to show, as we did above. The primary argument show.max_rows is to regulate the utmost variety of rows to show and 500 is the worth we set as the utmost row quantity.

Despite the fact that this methodology is broadly used, it’s not best to place it inside an executable Python file, particularly when you have a number of knowledge frames to print and they’re desired to show totally different numbers of rows.

For instance, I’ve a script structured as proven,

## Code Block 1 ##
...
print(df1.head(20))
...
## Code Block 2 ##
...
print(df2.head(100))
...
## Code Block N ##
...
print(df_n)
...

we’ve totally different numbers of high rows to indicate by means of your entire script, and typically we need to see your entire printed knowledge body, however typically we solely care concerning the dimension and construction of the information body with out the necessity to see your entire knowledge.

In such a case, we in all probability want to make use of the perform pd.set_option() to set the specified show or pd.reset_option() to make use of the default choices each time earlier than we print a knowledge body, which makes it very messy and troublesome.

## Code Block 1 ##
...
pd.set_option('show.max_rows', 20)
print(df1.head(20))
...
## Code Block 2 ##
...
pd.set_option('show.max_rows', 100)
print(df2.head(100))
...
## Code Block N ##
...
pd.reset_option('show.max_rows')
print(df_n)
...

There’s really a extra versatile and efficient means of exhibiting your entire knowledge body with out specifying the show choices for Pandas.

to_string()

to_string() instantly switch the pd.DataFrame object to a string object and after we print it out, it doesn’t care concerning the show restrict from pandas .

pd.set_option('show.max_rows', 10)
print(df.head(100).to_string())

We will see above that though I set the utmost variety of rows to show as 10, to_string() helps us print your entire knowledge body of 100 rows.

The perform, to_string() , converts a whole knowledge body to the string format, so it may hold all of the values and indexes within the knowledge body within the printing step. Since set_option() is just efficient on pandas objects, our printing string is just not restricted by the utmost variety of rows to show set earlier.

So, the technique is that you just don’t must set something by way of set_option() and also you solely want to make use of to_string() to see your entire knowledge body. It should prevent from occupied with which choice to set by which half throughout the script.

Takeaways

1. Use set_option('show.max_rows') when you have got a constant variety of rows to show throughout your entire script.
2. Use to_string() if you wish to print out your entire Pandas knowledge body it doesn’t matter what Pandas choices have been set.

Thanks for studying! Hope you take pleasure in utilizing the Pandas trick in your work!

Please subscribe to my Medium if you wish to learn extra tales from me. And you can even be part of the Medium membership by my referral hyperlink!