使用Xilinx BNN-PYNQ部署的Tiny YOLO示例。_Example of Tiny YOLO deploy

# MNIST Digit Classification [MNIST](http://yann.lecun.com/exdb/mnist/) is well-known dataset of handwritten digits. We'll use [LeNet-5](http://yann.lecun.com/exdb/lenet/)-like architecture for MNIST digits recognition task. LeNet-5 is proposed by Y.LeCun, which is known to work well on handwritten digit recognition. We replace LeNet-5's RBF layer with normal fully-connected layer. ## Prerequisites for this example - OpenCV ## Constructing Model Let's define the LeNet network. At first, you have to select loss-function and learning-algorithm by declaration of network class. Then, you can add layers from top to bottom by operator <<. ```cpp // specify loss-function and learning strategy network<mse, adagrad> nn; // connection table [Y.Lecun, 1998 Table.1] #define O true #define X false static const bool tbl [] = { O, X, X, X, O, O, O, X, X, O, O, O, O, X, O, O, O, O, X, X, X, O, O, O, X, X, O, O, O, O, X, O, O, O, O, X, X, X, O, O, O, X, X, O, X, O, O, O, X, O, O, O, X, X, O, O, O, O, X, X, O, X, O, O, X, X, O, O, O, X, X, O, O, O, O, X, O, O, X, O, X, X, X, O, O, O, X, X, O, O, O, O, X, O, O, O }; #undef O #undef X // construct nets nn << convolutional_layer<tan_h>(32, 32, 5, 1, 6) // C1, 1@32x32-in, 6@28x28-out << average_pooling_layer<tan_h>(28, 28, 6, 2) // S2, 6@28x28-in, 6@14x14-out << convolutional_layer<tan_h>(14, 14, 5, 6, 16, connection_table(tbl, 6, 16)) // C3, 6@14x14-in, 16@10x10-in << average_pooling_layer<tan_h>(10, 10, 16, 2) // S4, 16@10x10-in, 16@5x5-out << convolutional_layer<tan_h>(5, 5, 5, 16, 120) // C5, 16@5x5-in, 120@1x1-out << fully_connected_layer<tan_h>(120, 10); // F6, 120-in, 10-out ``` What does ```tbl``` mean? LeNet has "sparsity" between S2 and C3 layer. Specifically, each feature map in C3 is connected to a subset of S2's feature maps so that each feature maps get different set of inputs (and hopefully they become compelemtary feature extractor). tiny-cnn supports this sparsity by ```connection_table``` structure which parameters of constructor are ```bool``` table and number of in/out feature maps. ## Loading Dataset Tiny-cnn supports idx format, so all you have to do is calling parse_mnist_images and parse_mnist_labels functions. ```cpp // load MNIST dataset std::vector<label_t> train_labels, test_labels; std::vector<vec_t> train_images, test_images; parse_mnist_labels("train-labels.idx1-ubyte", &train_labels); parse_mnist_images("train-images.idx3-ubyte", &train_images, -1.0, 1.0, 2, 2); parse_mnist_labels("t10k-labels.idx1-ubyte", &test_labels); parse_mnist_images("t10k-images.idx3-ubyte", &test_images, -1.0, 1.0, 2, 2); ``` >Note: >Original MNIST images are 28x28 centered, [0,255]value. >This code rescale values [0,255] to [-1.0,1.0], and add 2px borders (so each image is 32x32). If you want to use another format for learning nets, see [Data Format](https://github.com/nyanp/tiny-cnn/wiki/Data-Format) page. # Defining Callback It's convenient if we can check recognition rate on test data, training time, and progress for each epoch while training. Tiny-cnn has callback mechanism for this purpose. We can use local variables(network, test-data, etc) in callback by using C++11's lambda. ```cpp boost::progress_display disp(train_images.size()); boost::timer t; // create callbacks auto on_enumerate_epoch = [&](){ std::cout << t.elapsed() << "s elapsed." << std::endl; tiny_cnn::result res = nn.test(test_images, test_labels); std::cout << res.num_success << "/" << res.num_total << std::endl; disp.restart(train_images.size()); t.restart(); }; auto on_enumerate_minibatch = [&](){ disp += minibatch_size; }; ``` ## Saving/Loading models Just use ```operator <<``` and ```operator >>``` with ostream/istream. ```cpp std::ofstream ofs("LeNet-weights"); ofs << nn; std::ifstream ifs("LeNet-weights"); ifs >> nn; ``` ## Putting it all together train.cpp ```cpp #include <iostream> #include "tiny_cnn/tiny_cnn.h" using namespace tiny_cnn; using namespace tiny_cnn::activation; void construct_net(network<mse, adagrad>& nn) { // connection table [Y.Lecun, 1998 Table.1] #define O true #define X false static const bool tbl[] = { O, X, X, X, O, O, O, X, X, O, O, O, O, X, O, O, O, O, X, X, X, O, O, O, X, X, O, O, O, O, X, O, O, O, O, X, X, X, O, O, O, X, X, O, X, O, O, O, X, O, O, O, X, X, O, O, O, O, X, X, O, X, O, O, X, X, O, O, O, X, X, O, O, O, O, X, O, O, X, O, X, X, X, O, O, O, X, X, O, O, O, O, X, O, O, O }; #undef O #undef X // construct nets nn << convolutional_layer<tan_h>(32, 32, 5, 1, 6) // C1, 1@32x32-in, 6@28x28-out << average_pooling_layer<tan_h>(28, 28, 6, 2) // S2, 6@28x28-in, 6@14x14-out << convolutional_layer<tan_h>(14, 14, 5, 6, 16, connection_table(tbl, 6, 16)) // C3, 6@14x14-in, 16@10x10-in << average_pooling_layer<tan_h>(10, 10, 16, 2) // S4, 16@10x10-in, 16@5x5-out << convolutional_layer<tan_h>(5, 5, 5, 16, 120) // C5, 16@5x5-in, 120@1x1-out << fully_connected_layer<tan_h>(120, 10); // F6, 120-in, 10-out } void train_lenet(std::string data_dir_path) { // specify loss-function and learning strategy network<mse, adagrad> nn; construct_net(nn); std::cout << "load models..." << std::endl; // load MNIST dataset std::vector<label_t> train_labels, test_labels; std::vector<vec_t> train_images, test_images; parse_mnist_labels(data_dir_path+"/train-labels.idx1-ubyte", &train_labels); parse_mnist_images(data_dir_path+"/train-images.idx3-ubyte", &train_images, -1.0, 1.0, 2, 2); parse_mnist_labels(data_dir_path+"/t10k-labels.idx1-ubyte", &test_labels); parse_mnist_images(data_dir_path+"/t10k-images.idx3-ubyte", &test_images, -1.0, 1.0, 2, 2); std::cout << "start training" << std::endl; progress_display disp(train_images.size()); timer t; int minibatch_size = 10; int num_epochs = 30; nn.optimizer().alpha *= std::sqrt(minibatch_size); // create callback auto on_enumerate_epoch = [&](){ std::cout << t.elapsed() << "s elapsed." << std::endl; tiny_cnn::result res = nn.test(test_images, test_labels); std::cout << res.num_success << "/" << res.num_total << std::endl; disp.restart(train_images.size()); t.restart(); }; auto on_enumerate_minibatch = [&](){ disp += minibatch_size; }; // training nn.train(train_images, train_labels, minibatch_size, num_epochs, on_enumerate_minibatch, on_enumerate_epoch); std::cout << "end training." << std::endl; // test and show results nn.test(test_images, test_labels).print_detail(std::cout); // save networks std::ofstream ofs("LeNet-weights"); ofs << nn; } int main(int argc, char **argv) { if (argc != 2) { std::cerr << "Usage : " << argv[0] << " path_to_data (example:../data)" << std::endl; return -1; } train_lenet(argv[1]); } ``` >Note: >Each image has 32x32 values, so dimension of first layer must be equal to 1024. You'll can get LeNet-Weights binary file after calling sample1_convnet() function. You can also download this file from [here](https://www.dropbox.com/s/mixgjhdi65jm7dl/LeNet-weights?dl=1). ## Use Learned Nets Here is an example of CUI-based OCR tool. test.cpp ```cpp #include <iostream> #include <opencv2/imgproc.hpp> #include <opencv2/imgcodecs.hpp> #include <opencv2/highgui.hpp> #include "tiny_cnn/tiny_cnn.h" using namespace tiny_cnn; using namespace tiny_cnn::activation; using namespace std; // rescale output to 0-100 template <typename Activation> double rescale(double x) { Activation a; return 100.0 * (x - a.scale().first) / (a.scale().second - a.scale().first); } // convert tiny_cnn::image to