General functions

Working with options

reset_option(key)	Reset one option to their default value.
get_option(key[, default])	Retrieves the value of the specified option.
set_option(key, value)	Sets the value of the specified option.
option_context(*args)	Context manager to temporarily set options in the with statement context.

Data manipulations and SQL

melt(frame[, id_vars, value_vars, var_name, …])	Unpivot a DataFrame from wide format to long format, optionally leaving identifier variables set.
merge(obj, right[, how, on, left_on, …])	Merge DataFrame objects with a database-style join.
merge_asof(left, right[, on, left_on, …])	Perform an asof merge.
get_dummies(data[, prefix, prefix_sep, …])	Convert categorical variable into dummy/indicator variables, also known as one hot encoding.
concat(objs[, axis, join, ignore_index, sort])	Concatenate pandas-on-Spark objects along a particular axis with optional set logic along the other axes.
sql(query[, index_col, args])	Execute a SQL query and return the result as a pandas-on-Spark DataFrame.
broadcast(obj)	Marks a DataFrame as small enough for use in broadcast joins.

Top-level missing data

isna(obj)	Detect missing values for an array-like object.
isnull(obj)	Detect missing values for an array-like object.
notna(obj)	Detect existing (non-missing) values.
notnull(obj)	Detect existing (non-missing) values.

Top-level dealing with numeric data

to_numeric(arg[, errors])

Convert argument to a numeric type.

Top-level dealing with datetimelike data

to_datetime(arg[, errors, format, unit, …])	Convert argument to datetime.
date_range([start, end, periods, freq, tz, …])	Return a fixed frequency DatetimeIndex.
to_timedelta(arg[, unit, errors])	Convert argument to timedelta.
timedelta_range([start, end, periods, freq, …])	Return a fixed frequency TimedeltaIndex, with day as the default frequency.

previous

pyspark.pandas.read_sql

next

pyspark.pandas.reset_option