🔗 GitHub
Documentation
Structures
DataSet
Structure to hold user training data and keep track of dimensions
typedef struct DataSet_ {
size_t rows;
size_t cols;
float** data;
} DataSet;
- rows
The number of rows in the dataset
- cols
The number of columns in the dataset
- data
The data held in the dataset, of the aforementioned dimensions
Matrix
Structure to hold internal network data and keep track of dimensions
typedef struct Matrix_ {
size_t rows;
size_t cols;
float* data;
} Matrix;
- rows
The number of rows in the matrix
- cols
The number of columns in the matrix
- data
The data held in the matrix, stored one-dimensionally in row-major order
Layer
Structure to represent a layer of a network and its data
typedef enum LAYER_TYPE_ {
INPUT,
HIDDEN,
OUTPUT
} LAYER_TYPE;
typedef struct Layer_ {
LAYER_TYPE type;
size_t size;
Activation activation;
Matrix* input;
} Layer;
- type
The type of layer (INPUT, OUTPUT, or HIDDEN)
- size
The number of neurons in the layer
- activation
The activation function for the layer
- input
The data currently stored in the layer in the form of a row vector with one entry per neuron in the layer
Connection
Structure to represent a link between two adjacent layers, as well the respective weights and bias
typedef struct Connection_ {
Layer* from;
Layer* to;
Matrix* weights;
Matrix* bias;
} Connection;
- from
The layer on the input side of the connection
- to
The layer on the output side of the connection
- weights
The weights matrix between the "from" and "to" layers, of dimensions "from"->size by "to"->size
- bias
The bias vector of the "to" layer, as a row vector of "to"->size
Network
Structure to represent a neural network, with its size, layers, and connections
typedef struct Network_ {
size_t numLayers;
Layer** layers;
size_t numConnections;
Connection** connections;
} Network;
- numLayers
The total number of layers in the network
- layers
The network's layers, in order
- numConnections
The total number of connections in the network
- connections
The network's connections, in order
Training Data Functions
createDataSet
Returns a pointer to a dataset of the given size and with the given data using the provided pointer
DataSet* createDataSet(size_t rows, size_t cols, float** data);
- rows
The number of rows in the dataset
- cols
The number of columns in the dataset
- data
The data, of the aforementioned dimensions, to be put into the dataset
createBatches
Returns a dataset split into a given number of batches, individually represented as dataset pointers
DataSet** createBatches(DataSet* allData, int numBatches);
- allData
The overall dataset to be split
- numBatches
The number of batches to split the dataset into
destroyDataSet
Frees a dataset and its data
void destroyDataSet(DataSet* dataset);
- dataset
The dataset to be freed
Network Functions
Activation
Functions that are applied to the input of each neuron within a layer
typedef void (*Activation)(Matrix*);
- sigmoid
Applies sigmoid function to layer (for hidden layers)
- relu
Applies ReLU function to layer (for hidden layers)
- tanH
Applies tanh function to layer (for hidden layers)
- softmax
Applies softmax function to layer (for output layer with CROSS_ENTROPY_LOSS)
- linear
Applies linear function to layer (for output layer with MEAN_SQUARED_ERROR)
createMatrix
Returns a pointer to a matrix of the given size and with the given data using the provided pointer
Matrix* createMatrix(size_t rows, size_t cols, float* data);
- rows
The number of rows in the matrix
- cols
The number of columns in the matrix
- data
The data, of the aforementioned dimensions, to be put into the matrix
destroyMatrix
Frees a matrix and its data
void destroyMatrix(Matrix* matrix);
- matrix
The matrix to free
createNetwork
Returns a pointer to a network with the given dimensions and functions
Network* createNetwork(size_t numFeatures, size_t numHiddenLayers, size_t*
hiddenSizes, Activation* hiddenActivations, size_t numOutputs,
Activation outputActivation);
- numFeatures
The number of neurons in the input layer
- numHiddenLayers
The number of hidden layers in the network
- hiddenSizes
The sizes of the hidden layers, provided as a list of sizes in order
- hiddenActivations
The activation functions of the hidden layers, provided as a list of functions, in order
- numOutputs
The number of neurons in the output layer
- outputActivation
The activation function of the output layer (should be softmax for classification, and linear for regression)
destroyNetwork
Frees a network, its connections, and its layers
void destroyNetwork(Network* network);
- network
The network to free
optimize
Trains a neural network given training parameters
typedef enum LOSS_FUNCTION_ {
CROSS_ENTROPY_LOSS,
MEAN_SQUARED_ERROR
} LOSS_FUNCTION;
typedef struct ParameterSet_ {
Network* network;
DataSet* data;
DataSet* classes;
LOSS_FUNCTION lossFunction;
size_t batchSize;
float learningRate;
float searchTime;
float regularizationStrength;
float momentumFactor;
int maxIters;
int shuffle;
int verbose;
} ParameterSet;
- network
The network to optimize
- data
The training data to train the network with
- classes
The target values for the training data
- lossFunction
The loss function to use when training, should be CROSS_ENTROPY_LOSS for classification and MEAN_SQUARED_ERROR for regression
- batchSize
The size of each batch (provide 1 for SGD, and data->rows for Batch)
- learningRate
The constant factor for initial learning rate
- searchTime
The parameter for "search-then-converge" learning rate annealing that describes when the learning rate will begin slowing down, provide 0 to not use annealing
- regularizationStrength
The constant factor for regularization
- momentumFactor
The constant factor for simple momentum (adds a fraction of the previous weight update)
- maxIters
The number of epochs to run the optimization algorithm for
- shuffle
If non-zero, will shuffle the data in between epochs (recommended)
- verbose
If non-zero, will print loss every 100 epochs
void optimize(ParameterSet params);
- params
The set of parameters to use when optimizing
forwardPass
Passes matrix input through a network, storing output in the network's last layer
void forwardPass(Network* network, Matrix* input);
- network
The network to use
- input
The data to pass through the network
forwardPassDataSet
Passes dataset input through a network, storing output in the network's last layer
void forwardPassDataSet(Network* network, DataSet* input);
- network
The network to use
- input
The data to pass through the network
getOutput
Returns the output in the last layer of a network (after using forwardPass)
Matrix* getOuput(Network* network);
- network
The network to use
predict
Returns an int* containing the indices of the predicted classes in the order they were given
int* predict(Network* network);
- network
The network to use
accuracy
Returns a float representing the network's accuracy on the provided data's predictions
float accuracy(Network* network, DataSet* data, DataSet* classes)
- network
The network to use
- data
The data to test on
- classes
The target data to test accuracy against
Serialization Functions
saveNetwork
Saves a network and all of its configuration details to a file
void saveNetwork(Network* network, char* path);
- network
The network to save
- path
The file path to where the network is to be saved
readNetwork
Returns a network created from the serialization of a previous network
Network* readNetwork(char* path);
- path
The file path to where the network had been saved