datasets の API リファレンス

datasets (huggingface/datasets) の公開 API 207 件 —— クラス 79、関数 36、メソッド 92。実際のソースを静的解析して抽出した正確なシグネチャを掲載しています。

種別	件数
クラス	79
関数	36
メソッド	92

API 一覧

classsrc.datasets.arrow_dataset.Column

An iterable for a specific column of a [`Dataset`].

classsrc.datasets.arrow_dataset.Dataset

A Dataset backed by an Arrow table.

method

src.datasets.arrow_dataset.Dataset.add_column(name:str, column:Union[list, np.ndarray], new_fingerprint:Optional[str]=None, feature:Optional[FeatureType]=None)

Add column to Dataset.

methodsrc.datasets.arrow_dataset.Dataset.add_item(item:dict, new_fingerprint:Optional[str]=None)

Add item to Dataset.

method

src.datasets.arrow_dataset.Dataset.cast_column(column:str, feature:FeatureType, new_fingerprint:Optional[str]=None) -> 'Dataset'

Cast column to feature for decoding.

methodsrc.datasets.arrow_dataset.Dataset.column_names() -> list[str]

Names of the columns in the dataset.

methodsrc.datasets.arrow_dataset.Dataset.data() -> Table

The Apache Arrow table backing the dataset.

methodsrc.datasets.arrow_dataset.Dataset.flatten(new_fingerprint:Optional[str]=None, max_depth=16) -> 'Dataset'

Flatten the table.

methodsrc.datasets.arrow_dataset.Dataset.num_columns() -> int

Number of columns in the dataset.

methodsrc.datasets.arrow_dataset.Dataset.skip(n:int) -> 'Dataset'

Create a new [`Dataset`] that skips the first `n` elements.

methodsrc.datasets.arrow_dataset.Dataset.take(n:int) -> 'Dataset'

Create a new [`Dataset`] with only the first `n` elements.

method

src.datasets.arrow_dataset.Dataset.to_dict(batch_size:Optional[int]=None, batched:bool=False) -> Union[dict, Iterator[dict]]

Returns the dataset as a Python dict.

methodsrc.datasets.arrow_dataset.Dataset.to_list() -> list

Returns the dataset as a Python list.

method

src.datasets.arrow_dataset.Dataset.to_pandas(batch_size:Optional[int]=None, batched:bool=False) -> Union[pd.DataFrame, Iterator[pd.DataFrame]]

Returns the dataset as a `pandas.DataFrame`.

methodsrc.datasets.arrow_dataset.Dataset.unique(column:str) -> list

Return a list of the unique elements in a column.

classsrc.datasets.arrow_dataset.NonExistentDatasetError

Used when we expect the existence of a dataset

classsrc.datasets.arrow_reader.ArrowReader

Build a Dataset object out of Instruction instance(s).

methodsrc.datasets.arrow_reader.ArrowReader.read_table(filename, in_memory=False) -> Table

Read table from file.

classsrc.datasets.arrow_reader.BaseReader

Build a Dataset object out of Instruction instance(s).

methodsrc.datasets.arrow_reader.BaseReader.read(name, instructions, split_infos, in_memory=False)

Returns Dataset instance(s).

classsrc.datasets.arrow_reader.ParquetReader

Build a Dataset object out of Instruction instance(s).

classsrc.datasets.arrow_reader.ReadInstruction

Reading instruction for a dataset.

classsrc.datasets.arrow_writer.ArrowWriter

Shuffles and writes Examples to Arrow files.

methodsrc.datasets.arrow_writer.ArrowWriter.write_table(pa_table:pa.Table, writer_batch_size:Optional[int]=None)

Write a Table to file.

classsrc.datasets.builder.BuilderConfig

Base class for `DatasetBuilder` data configuration.

methodsrc.datasets.builder.BuilderConfig.create_config_id(config_kwargs:dict, custom_features:Optional[Features]=None) -> str

The config id is used to build the cache directory.

classsrc.datasets.builder.DatasetBuilder

Abstract base class for all datasets.

method

src.datasets.builder.DatasetBuilder.as_dataset(split:Optional[Union[str, Split, list[str], list[Split]]]=None, in_memory=False) -> Union[Dataset, DatasetDict]

Return a Dataset for the specified split.

classsrc.datasets.data_files.DataFilesList

List of data files (absolute local paths or URLs).

classsrc.datasets.download.download_config.DownloadConfig

Configuration for our cached path manager.

classsrc.datasets.exceptions.DataFilesNotFoundError

No (supported) data files found.

classsrc.datasets.exceptions.DatasetNotFoundError

Dataset not found.

classsrc.datasets.exceptions.DatasetsError

Base class for exceptions in this library.

classsrc.datasets.exceptions.DefunctDatasetError

The dataset has been defunct.

classsrc.datasets.exceptions.ExpectedMoreSplitsError

Some recorded splits are missing.

classsrc.datasets.exceptions.FileNotFoundDatasetsError

FileNotFoundError raised by this library.

classsrc.datasets.exceptions.SplitsVerificationError

Error raised during splits verifications.

classsrc.datasets.exceptions.UnexpectedDownloadedFileError

Some downloaded files were not expected.

classsrc.datasets.exceptions.UnexpectedSplitsError

The expected splits of the downloaded file is missing.

classsrc.datasets.features.audio.Audio

Audio [`Feature`] to extract audio data from an audio file.

methodsrc.datasets.features.audio.Audio.cast_storage(storage:Union[pa.StringArray, pa.StructArray]) -> pa.StructArray

Cast an Arrow array to the Audio arrow storage type.

method

src.datasets.features.audio.Audio.decode_example(value:dict, token_per_repo_id:Optional[dict[str, Union[str, bool, None]]]=None) -> 'AudioDecoder'

Decode example audio file into audio data.

method

src.datasets.features.audio.Audio.embed_storage(storage:pa.StructArray, token_per_repo_id=None, local_files:bool=True, remote_files:bool=True) -> pa.StructArray

Embed audio files into the Arrow array.

methodsrc.datasets.features.audio.Audio.encode_example(value:Union[str, bytes, bytearray, dict, 'AudioDecoder']) -> dict

Encode example into a format for Arrow.

classsrc.datasets.features.features.Array2D

Create a two-dimensional array.

classsrc.datasets.features.features.Array3D

Create a three-dimensional array.

classsrc.datasets.features.features.Array4D

Create a four-dimensional array.

classsrc.datasets.features.features.Array5D

Create a five-dimensional array.

classsrc.datasets.features.features.ClassLabel

Feature type for integer class labels.

methodsrc.datasets.features.features.ClassLabel.int2str(values:Union[int, Iterable]) -> Union[str, Iterable]

Conversion `integer` => class name `string`.

methodsrc.datasets.features.features.ClassLabel.str2int(values:Union[str, Iterable]) -> Union[int, Iterable]

Conversion class name `string` => `integer`.

methodsrc.datasets.features.features.Features.arrow_schema()

Features schema.

methodsrc.datasets.features.features.Features.copy() -> 'Features'

Make a deep copy of [`Features`].

methodsrc.datasets.features.features.Features.encode_column(column, column_name:str)

Encode column into a format for Arrow.

methodsrc.datasets.features.features.Features.flatten(max_depth=16) -> 'Features'

Flatten the features.

methodsrc.datasets.features.features.Features.from_arrow_schema(pa_schema:pa.Schema) -> 'Features'

Construct [`Features`] from Arrow Schema.

methodsrc.datasets.features.features.Features.from_dict(dic) -> 'Features'

Construct [`Features`] from dict.

methodsrc.datasets.features.features.Features.type()

Features field types.

classsrc.datasets.features.features.Json

Feature type for JSON objects.

methodsrc.datasets.features.features.Json.cast_storage(storage:Union[pa.Array]) -> pa.JsonArray

Cast an Arrow array to the `Json` arrow storage type.

classsrc.datasets.features.features.Value

Scalar feature value of a particular data type.

func

src.datasets.features.features.decode_nested_example(schema, obj, token_per_repo_id:Optional[dict[str, Union[str, bool, None]]]=None)

Decode a nested example.

funcsrc.datasets.features.features.encode_nested_example(schema, obj, level=0)

Encode a nested example.

funcsrc.datasets.features.features.require_decoding(feature:FeatureType, ignore_decode_attribute:bool=False) -> bool

Check if a (possibly nested) feature requires decoding.

funcsrc.datasets.features.features.to_pyarrow_listarray(data:Any, pa_type:_ArrayXDExtensionType) -> pa.Array

Convert to PyArrow ListArray.

classsrc.datasets.features.image.Image

Image [`Feature`] to read image data from an image file.

methodsrc.datasets.features.image.Image.decode_example(value:dict, token_per_repo_id=None) -> 'PIL.Image.Image'

Decode example image file into image data.

method

src.datasets.features.image.Image.embed_storage(storage:pa.StructArray, token_per_repo_id=None, local_files:bool=True, remote_files:bool=True) -> pa.StructArray

Embed image files into the Arrow array.

method

src.datasets.features.image.Image.encode_example(value:Union[str, bytes, bytearray, dict, np.ndarray, 'PIL.Image.Image']) -> dict

Encode example into a format for Arrow.

classsrc.datasets.features.mesh.Mesh

Mesh [`Feature`] to read 3D mesh data from a file.

methodsrc.datasets.features.mesh.Mesh.cast_storage(storage:Union[pa.StringArray, pa.StructArray]) -> pa.StructArray

Cast an Arrow array to the Mesh arrow storage type.

method

src.datasets.features.mesh.Mesh.decode_example(value:dict, token_per_repo_id=None) -> Union['trimesh.Trimesh', 'trimesh.Scene']

Decode example mesh file.

method

src.datasets.features.mesh.Mesh.embed_storage(storage:pa.StructArray, token_per_repo_id=None, local_files:bool=True, remote_files:bool=True) -> pa.StructArray

Embed mesh files into the Arrow array.

method

src.datasets.features.mesh.Mesh.encode_example(value:Union[str, bytes, bytearray, dict, 'trimesh.Trimesh', 'trimesh.Scene']) -> dict

Encode example into a format for Arrow.

method

src.datasets.features.pdf.Pdf.cast_storage(storage:Union[pa.StringArray, pa.StructArray, pa.ListArray]) -> pa.StructArray

Cast an Arrow array to the Pdf arrow storage type.

methodsrc.datasets.features.pdf.Pdf.decode_example(value:dict, token_per_repo_id=None) -> 'pdfplumber.pdf.PDF'

Decode example pdf file into pdf data.

method

src.datasets.features.pdf.Pdf.embed_storage(storage:pa.StructArray, token_per_repo_id=None, local_files:bool=True, remote_files:bool=True) -> pa.StructArray

Embed PDF files into the Arrow array.

methodsrc.datasets.features.pdf.Pdf.encode_example(value:Union[str, bytes, bytearray, dict, 'pdfplumber.pdf.PDF']) -> dict

Encode example into a format for Arrow.

funcsrc.datasets.features.pdf.encode_pdfplumber_pdf(pdf:'pdfplumber.pdf.PDF') -> dict

Encode a pdfplumber.pdf.PDF object into a dictionary.

funcsrc.datasets.features.pdf.pdf_to_bytes(pdf:'pdfplumber.pdf.PDF') -> bytes

Convert a pdfplumber.pdf.PDF object to bytes.

classsrc.datasets.features.video.Video

Video [`Feature`] to read video data from a video file.

method

src.datasets.features.video.Video.decode_example(value:Union[str, Example], token_per_repo_id:Optional[dict[str, Union[bool, str]]]=None) -> 'VideoDecoder'

Decode example video file into video data.

method

src.datasets.features.video.Video.embed_storage(storage:pa.StructArray, token_per_repo_id=None, local_files:bool=True, remote_files:bool=True) -> pa.StructArray

Embed image files into the Arrow array.

method

src.datasets.features.video.Video.encode_example(value:Union[str, bytes, bytearray, Example, np.ndarray, 'VideoDecoder']) -> Example

Encode example into a format for Arrow.

funcsrc.datasets.filesystems.is_remote_filesystem(fs:fsspec.AbstractFileSystem) -> bool

Checks if `fs` is a remote filesystem.

funcsrc.datasets.filesystems.rename(fs:fsspec.AbstractFileSystem, src:str, dst:str)

Renames the file `src` in `fs` to `dst`.

classsrc.datasets.fingerprint.Hasher

Hasher that accepts python objects as inputs.

classsrc.datasets.formatting.formatting.LazyDict

A dictionary backed by Arrow data.

classsrc.datasets.info.DatasetInfo

Information about a dataset.

classsrc.datasets.info.NonMatchingCachedSizesError

The prepared split doesn't have expected sizes.

classsrc.datasets.io.spark.SparkDatasetReader

A dataset reader that reads from a Spark DataFrame.

classsrc.datasets.iterable_dataset.IterableDataset

A Dataset backed by an iterable.

methodsrc.datasets.iterable_dataset.IterableDataset.add_column(name:str, column:Union[list, np.array]) -> 'IterableDataset'

Add column to Dataset.

methodsrc.datasets.iterable_dataset.IterableDataset.cast(features:Features) -> 'IterableDataset'

Cast the dataset to a new set of features.

methodsrc.datasets.iterable_dataset.IterableDataset.cast_column(column:str, feature:FeatureType) -> 'IterableDataset'

Cast column to feature for decoding.

methodsrc.datasets.iterable_dataset.IterableDataset.column_names() -> Optional[list[str]]

Names of the columns in the dataset.

methodsrc.datasets.iterable_dataset.IterableDataset.load_state_dict(state_dict:dict) -> None

Load the state_dict of the dataset.

methodsrc.datasets.iterable_dataset.IterableDataset.num_columns() -> Optional[int]

Number of columns in the dataset.

methodsrc.datasets.iterable_dataset.IterableDataset.reshard() -> 'IterableDataset'

Reshard the dataset if possible, i.e.

methodsrc.datasets.iterable_dataset.IterableDataset.state_dict() -> dict

Get the current state_dict of the dataset.

method

src.datasets.iterable_dataset.IterableDataset.to_csv(path_or_buf:Union[PathLike, BinaryIO], batch_size:Optional[int]=None, storage_options:Optional[dict]=None, **to_csv_kwargs) -> int

Exports the dataset to csv.

method

src.datasets.iterable_dataset.IterableDataset.to_dict(batch_size:Optional[int]=None, batched:bool=False) -> Union[dict, Iterator[dict]]

Returns the dataset as a Python dict.

methodsrc.datasets.iterable_dataset.IterableDataset.to_list() -> list

Returns the dataset as a Python list.

methodsrc.datasets.iterable_dataset.IterableDataset.with_format(type:Optional[str]=None) -> 'IterableDataset'

Return a dataset with the specified format.

funcsrc.datasets.load.increase_load_count(name:str)

Update the download count of a dataset.

classsrc.datasets.packaged_modules.arrow.arrow.ArrowConfig

BuilderConfig for Arrow.

classsrc.datasets.packaged_modules.audiofolder.audiofolder.AudioFolderConfig

Builder Config for AudioFolder.

classsrc.datasets.packaged_modules.conll.conll.ConllConfig

BuilderConfig for CoNLL-style files.

classsrc.datasets.packaged_modules.csv.csv.CsvConfig

BuilderConfig for CSV.

classsrc.datasets.packaged_modules.folder_based_builder.folder_based_builder.FolderBasedBuilderConfig

BuilderConfig for AutoFolder.

classsrc.datasets.packaged_modules.hdf5.hdf5.HDF5Config

BuilderConfig for HDF5.

classsrc.datasets.packaged_modules.iceberg.iceberg.IcebergConfig

BuilderConfig for Apache Iceberg format.

classsrc.datasets.packaged_modules.imagefolder.imagefolder.ImageFolderConfig

BuilderConfig for ImageFolder.

classsrc.datasets.packaged_modules.json.json.JsonConfig

BuilderConfig for JSON.

classsrc.datasets.packaged_modules.lance.lance.LanceConfig

BuilderConfig for Lance format.

classsrc.datasets.packaged_modules.meshfolder.meshfolder.MeshFolderConfig

BuilderConfig for MeshFolder.

classsrc.datasets.packaged_modules.niftifolder.niftifolder.NiftiFolderConfig

BuilderConfig for NiftiFolder.

classsrc.datasets.packaged_modules.pandas.pandas.PandasConfig

BuilderConfig for Pandas.

classsrc.datasets.packaged_modules.parquet.parquet.ParquetConfig

BuilderConfig for Parquet.

classsrc.datasets.packaged_modules.pdffolder.pdffolder.PdfFolderConfig

BuilderConfig for ImageFolder.

classsrc.datasets.packaged_modules.spark.spark.SparkConfig

BuilderConfig for Spark.

classsrc.datasets.packaged_modules.sql.sql.SqlConfig

BuilderConfig for SQL.

classsrc.datasets.packaged_modules.text.text.TextConfig

BuilderConfig for text files.

classsrc.datasets.packaged_modules.videofolder.videofolder.VideoFolderConfig

BuilderConfig for ImageFolder.

funcsrc.datasets.packaged_modules.webdataset._tenbin.str64(s)

Convert a string to an int64.

funcsrc.datasets.packaged_modules.webdataset._tenbin.unstr64(i)

Convert an int64 to a string.

classsrc.datasets.packaged_modules.xml.xml.XmlConfig

BuilderConfig for xml files.

classsrc.datasets.search.BaseIndex

Base class for indexing

methodsrc.datasets.search.BaseIndex.load(file:Union[str, PurePath]) -> 'BaseIndex'

Deserialize the index from disk

methodsrc.datasets.search.BaseIndex.save(file:Union[str, PurePath])

Serialize the index on disk

methodsrc.datasets.search.BaseIndex.search(query, k:int=10, **kwargs) -> SearchResults

To implement.

methodsrc.datasets.search.BaseIndex.search_batch(queries, k:int=10, **kwargs) -> BatchedSearchResults

Find the nearest examples indices to the query.

classsrc.datasets.search.ElasticSearchIndex

Sparse index using Elasticsearch.

methodsrc.datasets.search.ElasticSearchIndex.add_documents(documents:Union[list[str], 'Dataset'], column:Optional[str]=None)

Add documents to the index.

methodsrc.datasets.search.ElasticSearchIndex.search(query:str, k=10, **kwargs) -> SearchResults

Find the nearest examples indices to the query.

classsrc.datasets.search.FaissIndex

Dense index using Faiss.

method

src.datasets.search.FaissIndex.load(file:Union[str, PurePath], device:Optional[Union[int, list[int]]]=None, storage_options:Optional[dict]=None) -> 'FaissIndex'

Deserialize the FaissIndex from disk

methodsrc.datasets.search.FaissIndex.save(file:Union[str, PurePath], storage_options:Optional[dict]=None)

Serialize the FaissIndex on disk

methodsrc.datasets.search.FaissIndex.search(query:np.array, k=10, **kwargs) -> SearchResults

Find the nearest examples indices to the query.

methodsrc.datasets.search.FaissIndex.search_batch(queries:np.array, k=10, **kwargs) -> BatchedSearchResults

Find the nearest examples indices to the queries.

classsrc.datasets.search.IndexableMixin

Add indexing features to `datasets.Dataset`

methodsrc.datasets.search.IndexableMixin.drop_index(index_name:str)

Drop the index with the specified column.

methodsrc.datasets.search.IndexableMixin.get_index(index_name:str) -> BaseIndex

List the `index_name`/identifiers of all the attached indexes.

method

src.datasets.search.IndexableMixin.save_faiss_index(index_name:str, file:Union[str, PurePath], storage_options:Optional[dict]=None)

Save a FaissIndex on disk.

classsrc.datasets.splits.NamedSplit

Descriptor corresponding to a named split (train, test, ...).

classsrc.datasets.splits.Split

`Enum` for dataset splits.

classsrc.datasets.splits.SplitBase

Abstract base class for Split compositionality.

classsrc.datasets.splits.SplitDict

Split info object.

methodsrc.datasets.splits.SplitDict.add(split_info:SplitInfo)

Add the split info.

classsrc.datasets.splits.SplitGenerator

Defines the split information for the generator.

classsrc.datasets.splits.SplitReadInstruction

Object containing the reading instruction for the dataset.

classsrc.datasets.splits.SubSplitInfo

Wrapper around a sub split info.

classsrc.datasets.table.InMemoryTable

The table is said in-memory when it is loaded into the user's RAM.

methodsrc.datasets.table.InMemoryTable.add_column(*args, **kwargs)

Add column to Table at position.

methodsrc.datasets.table.InMemoryTable.filter(*args, **kwargs)

Select records from a Table.

methodsrc.datasets.table.InMemoryTable.flatten(*args, **kwargs)

Flatten this Table.

methodsrc.datasets.table.InMemoryTable.from_arrays(*args, **kwargs)

Construct a Table from Arrow arrays.

methodsrc.datasets.table.InMemoryTable.select(*args, **kwargs)

Select columns of the table.

classsrc.datasets.table.Table

Wraps a pyarrow Table by using composition.

methodsrc.datasets.table.Table.add_column(*args, **kwargs)

Add column to Table at position.

methodsrc.datasets.table.Table.append_column(*args, **kwargs)

Append column at end of columns.

methodsrc.datasets.table.Table.cast(*args, **kwargs)

Cast table values to another schema.

methodsrc.datasets.table.Table.column_names()

Names of the table's columns.

methodsrc.datasets.table.Table.columns()

List of all columns in numerical order.

methodsrc.datasets.table.Table.filter(*args, **kwargs)

Select records from a Table.

methodsrc.datasets.table.Table.flatten(*args, **kwargs)

Flatten this Table.

methodsrc.datasets.table.Table.num_columns()

Number of columns in this table.

methodsrc.datasets.table.Table.num_rows()

Number of rows in this table.

methodsrc.datasets.table.Table.schema()

Schema of the table and its columns.

methodsrc.datasets.table.Table.select(*args, **kwargs)

Select columns of the table.

methodsrc.datasets.table.Table.set_column(*args, **kwargs)

Replace column in Table at position.

methodsrc.datasets.table.Table.slice(*args, **kwargs)

Compute zero-copy slice of this Table.

methodsrc.datasets.table.Table.to_reader(max_chunksize:Optional[int]=None)

Convert the Table to a RecordBatchReader.

methodsrc.datasets.table.Table.validate(*args, **kwargs)

Perform validation checks.

funcsrc.datasets.table.cast_table_to_schema(table:pa.Table, schema:pa.Schema)

Cast a table to the arrow schema.

funcsrc.datasets.table.concat_tables(tables:list[Table], axis:int=0) -> Table

Concatenate tables.

funcsrc.datasets.table.list_table_cache_files(table:Table) -> list[str]

Get the cache files that are loaded by the table.

funcsrc.datasets.table.table_cast(table:pa.Table, schema:pa.Schema)

Improved version of `pa.Table.cast`.

funcsrc.datasets.table.table_flatten(table:pa.Table)

Improved version of `pa.Table.flatten`.

funcsrc.datasets.table.table_iter(table:Table, batch_size:int, drop_last_batch=False) -> Iterator[pa.Table]

Iterate over sub-tables of size `batch_size`.

classsrc.datasets.utils._dataset_viewer.DatasetViewerError

Dataset viewer error.

funcsrc.datasets.utils._dill.dump(obj, file)

Pickle an object to a file.

funcsrc.datasets.utils._dill.dumps(obj)

Pickle an object to a string.

classsrc.datasets.utils._filelock.FileLock

A `filelock.FileLock` initializer that handles long paths.

funcsrc.datasets.utils.experimental.experimental(fn:Callable) -> Callable

Decorator to flag a feature as experimental.

classsrc.datasets.utils.file_utils.FilesIterable

An iterable of paths from a list of directories or files

funcsrc.datasets.utils.file_utils.get_authentication_headers_for_url(url:str, token:Optional[Union[str, bool]]=None) -> dict

Handle the HF authentication

funcsrc.datasets.utils.file_utils.relative_to_absolute_path(path:T) -> T

Convert relative path to absolute path.

funcsrc.datasets.utils.file_utils.xgetsize(path, download_config:Optional[DownloadConfig]=None) -> int

Extend `os.path.getsize` function to support remote files.

funcsrc.datasets.utils.file_utils.xisdir(path, download_config:Optional[DownloadConfig]=None) -> bool

Extend `os.path.isdir` function to support remote files.

funcsrc.datasets.utils.file_utils.xisfile(path, download_config:Optional[DownloadConfig]=None) -> bool

Extend `os.path.isfile` function to support remote files.

funcsrc.datasets.utils.file_utils.xlistdir(path:str, download_config:Optional[DownloadConfig]=None) -> list[str]

Extend `os.listdir` function to support remote files.

classsrc.datasets.utils.info_utils.VerificationMode

`Enum` that specifies which verification checks to run.

funcsrc.datasets.utils.info_utils.get_size_checksum_dict(path:str, record_checksum:bool=False) -> dict

Compute the file size and the sha256 checksum of a file

funcsrc.datasets.utils.logging.disable_propagation() -> None

Disable propagation of the library log outputs.

funcsrc.datasets.utils.logging.enable_propagation() -> None

Enable propagation of the library log outputs.

funcsrc.datasets.utils.logging.get_logger(name:Optional[str]=None) -> logging.Logger

Return a logger with the specified name.

funcsrc.datasets.utils.logging.set_verbosity(verbosity:int) -> None

Set the level for the Hugging Face Datasets library's root logger.

classsrc.datasets.utils.metadata.MetadataConfigs

Should be in format {config_name: {**config_params}}.

classsrc.datasets.utils.py_utils.NonMutableDict

Dict where keys can only be added but not modified.

classsrc.datasets.utils.py_utils.classproperty

Descriptor to be used as decorator for @classmethods.

funcsrc.datasets.utils.py_utils.string_to_dict(string:str, pattern:str) -> Optional[dict[str, str]]

Un-format a string using a python f-string pattern.

funcsrc.datasets.utils.py_utils.temp_seed(seed:int, set_pytorch=False, set_tensorflow=False)

Temporarily set the random seed.

classsrc.datasets.utils.version.Version

Dataset version `MAJOR.MINOR.PATCH`.

funcutils.release.get_version()

Reads the current version in the __init__.

funcutils.release.global_version_update(version)

Update the version in all needed files.

funcutils.release.post_release_work()

Do all the necesarry post-release steps.

この情報について

掲載しているシグネチャは huggingface/datasets の公開ソースコードを Python の ast モジュールで静的解析し、引数名・デフォルト値・型注釈・戻り値型をそのまま抽出したものです。実装コードは保存していません。詳しくは仕組みの解説をご覧ください。

収録ライブラリ一覧（全 805 件）へ戻る