#include <FullyConnected.h>

Inheritance diagram for MiniDNN::FullyConnected< Activation >:

Public Member Functions
	FullyConnected (const int in_size, const int out_size)

void	init (const Scalar &mu, const Scalar &sigma, RNG &rng)

void	forward (const Matrix &prev_layer_data)

const Matrix &	output () const

void	backprop (const Matrix &prev_layer_data, const Matrix &next_layer_data)

const Matrix &	backprop_data () const

void	update (Optimizer &opt)

std::vector< Scalar >	get_parameters () const

void	set_parameters (const std::vector< Scalar > &param)

std::vector< Scalar >	get_derivatives () const

Public Member Functions inherited from MiniDNN::Layer
	Layer (const int in_size, const int out_size)

virtual	~Layer ()

int	in_size () const

int	out_size () const

Detailed Description

template<typename Activation>
class MiniDNN::FullyConnected< Activation >

Fully connected hidden layer

Definition at line 20 of file FullyConnected.h.

Constructor & Destructor Documentation

◆ FullyConnected()

template<typename Activation >

MiniDNN::FullyConnected< Activation >::FullyConnected	(	const int	in_size,
		const int	out_size
	)

inline

Constructor

Parameters

in_size	Number of input units.
out_size	Number of output units.

Definition at line 44 of file FullyConnected.h.

Member Function Documentation

◆ init()

template<typename Activation >

void MiniDNN::FullyConnected< Activation >::init	(	const Scalar &	mu,
		const Scalar &	sigma,
		RNG &	rng
	)

inlinevirtual

Initialize layer parameters using \(N(\mu, \sigma^2)\) distribution

Parameters

mu	Mean of the normal distribution.
sigma	Standard deviation of the normal distribution.
rng	The random number generator of type RNG.

Implements MiniDNN::Layer.

Definition at line 48 of file FullyConnected.h.

◆ forward()

template<typename Activation >

void MiniDNN::FullyConnected< Activation >::forward ( const Matrix & prev_layer_data )

inlinevirtual

Compute the output of this layer

The purpose of this function is to let the hidden layer compute information that will be passed to the next layer as the input. The concrete behavior of this function is subject to the implementation, with the only requirement that after calling this function, the Layer::output() member function will return a reference to the output values.

Parameters

prev_layer_data The output of previous layer, which is also the input of this layer. prev_layer_data should have in_size rows as in the constructor, and each column of prev_layer_data is an observation.

Implements MiniDNN::Layer.

Definition at line 61 of file FullyConnected.h.

◆ output()

template<typename Activation >

const Matrix& MiniDNN::FullyConnected< Activation >::output ( ) const

inlinevirtual

Obtain the output values of this layer

This function is assumed to be called after Layer::forward() in each iteration. The output are the values of output hidden units after applying activation function. The main usage of this function is to provide the prev_layer_data parameter in Layer::forward() of the next layer.

Returns: A reference to the matrix that contains the output values. The matrix should have out_size rows as in the constructor, and have number of columns equal to that of prev_layer_data in the Layer::forward() function. Each column represents an observation.

Implements MiniDNN::Layer.

Definition at line 74 of file FullyConnected.h.

◆ backprop()

template<typename Activation >

void MiniDNN::FullyConnected< Activation >::backprop	(	const Matrix &	prev_layer_data,
		const Matrix &	next_layer_data
	)

inlinevirtual

Compute the gradients of parameters and input units using back-propagation

The purpose of this function is to compute the gradient of input units, which can be retrieved by Layer::backprop_data(), and the gradient of layer parameters, which could later be used by the Layer::update() function.

Parameters

prev_layer_data	The output of previous layer, which is also the input of this layer. `prev_layer_data` should have `in_size` rows as in the constructor, and each column of `prev_layer_data` is an observation.
next_layer_data	The gradients of the input units of the next layer, which is also the gradients of the output units of this layer. `next_layer_data` should have `out_size` rows as in the constructor, and the same number of columns as `prev_layer_data`.

Implements MiniDNN::Layer.

Definition at line 81 of file FullyConnected.h.

◆ backprop_data()

template<typename Activation >

const Matrix& MiniDNN::FullyConnected< Activation >::backprop_data ( ) const

inlinevirtual

Obtain the gradient of input units of this layer

This function provides the next_layer_data parameter in Layer::backprop() of the previous layer, since the derivative of the input of this layer is also the derivative of the output of previous layer.

Implements MiniDNN::Layer.

Definition at line 104 of file FullyConnected.h.

◆ update()

template<typename Activation >

void MiniDNN::FullyConnected< Activation >::update ( Optimizer & opt )

inlinevirtual

Update parameters after back-propagation

Parameters

opt	The optimization algorithm to be used. See the Optimizer class.

Implements MiniDNN::Layer.

Definition at line 109 of file FullyConnected.h.

◆ get_parameters()

template<typename Activation >

std::vector<Scalar> MiniDNN::FullyConnected< Activation >::get_parameters ( ) const

inlinevirtual

Get serialized values of parameters

Implements MiniDNN::Layer.

Definition at line 120 of file FullyConnected.h.

◆ set_parameters()

template<typename Activation >

void MiniDNN::FullyConnected< Activation >::set_parameters ( const std::vector< Scalar > & param )

inlinevirtual

Set the values of layer parameters from serialized data

Reimplemented from MiniDNN::Layer.

Definition at line 130 of file FullyConnected.h.

◆ get_derivatives()

template<typename Activation >

std::vector<Scalar> MiniDNN::FullyConnected< Activation >::get_derivatives ( ) const

inlinevirtual

Get serialized values of the gradient of parameters

Implements MiniDNN::Layer.

Definition at line 139 of file FullyConnected.h.

The documentation for this class was generated from the following file:

Layer/FullyConnected.h

Public Member Functions

Detailed Description

template<typename Activation> class MiniDNN::FullyConnected< Activation >

Constructor & Destructor Documentation

◆ FullyConnected()

Member Function Documentation

◆ init()

◆ forward()

◆ output()

◆ backprop()

◆ backprop_data()

◆ update()

◆ get_parameters()

◆ set_parameters()

◆ get_derivatives()

template<typename Activation>
class MiniDNN::FullyConnected< Activation >