demo mxnet

demo/mxnet/TUTORIAL_CN.md

@@ -0,0 +1,60 @@
+# 在MXNet中使用VisualDL
+
+下面我们演示一下如何在MXNet中使用VisualDL，从而可以把MXNet的训练过程以及最后的模型可视化出来。我们将以MXNet用卷积神经网络(CNN, Convolutional Neural Network)来训练[MNIST](http://yann.lecun.com/exdb/mnist/)数据集作为例子。
+
+- [安装MXNet](#%E5%AE%89%E8%A3%85mxnet)
+- [安装VisualDL](#%E5%AE%89%E8%A3%85visualdl)
+- [开始编写训练MNIST的程序](#%E5%BC%80%E5%A7%8B%E7%BC%96%E5%86%99%E8%AE%AD%E7%BB%83mnist%E7%9A%84%E7%A8%8B%E5%BA%8F)
+
+## 安装MXNet
+请按照MXNet的[官方网站](https://mxnet.incubator.apache.org/get_started/install.html)来安装MXNet，并验证安装成功。
+
+
+    >>> import mxnet as mx
+    >>> a = mx.nd.ones((2, 3))
+    >>> b = a * 2 + 1
+    >>> b.asnumpy()
+    array([[ 3.,  3.,  3.],
+           [ 3.,  3.,  3.]], dtype=float32)
+
+## 安装VisualDL
+VisualDL的安装很简单。请按照VisualDL的[官方网站](https://github.com/PaddlePaddle/VisualDL)进行安装。具体只需要两步
+
+```
+python setup.py bdist_wheel
+pip install --upgrade dist/visualdl-0.0.1-py2-none-any.whl
+```
+
+## 开始编写训练MNIST的程序
+
+我们为您提供了一个演示程序 [mxnet_demo.py](./mxnet_demo.py)。里面展示了如何下载MNIST数据集以及编写MXNet程序来进行CNN的训练。MXNet的部分借鉴了MXNet[官方入门文件](https://mxnet.incubator.apache.org/tutorials/python/mnist.html)
+为了嵌入VisualDL程序，以便在MXNet训练时进行检测，我们需要声明一个LogWriter实例：
+
+```
+logger = LogWriter(logdir, sync_cycle=10)
+```
+
+logger实例里面包含VisualDL的四个功能模块 Scalar， Image 以及 Histogram。这里我们使用 Scalar 模块：
+
+```
+scalar0 = logger.scalar("scalars/scalar0")
+```
+
+模块的命名可以有 '/', 以便对于复杂模型创建不同的命名空间。
+
+MXNet在fit函数中提供了很多[API](https://mxnet.incubator.apache.org/api/python/index.html)。我们把自己编写好的回调函数 add_scalar 插入到相应的 API中
+
+```
+lenet_model.fit(train_iter,
+                eval_data=val_iter,
+                optimizer='sgd',
+                optimizer_params={'learning_rate':0.1},
+                eval_metric='acc',
+                # 在此嵌入我们自定义的回调函数
+                batch_end_callback=[add_scalar()],
+                num_epoch=2)
+```
+
+这样就好了。在MXNet的训练过程中，每一个批次（batch）训练完后，都会调用我们的回调函数来对准确率进行记录。如您所料，随着训练的进行，准确率会不断上升直到95%以上。以下是两个epoch训练过后的准确率走向：
+
+<p align=center><img width="80%" src="./epoch2_small.png" /></p>

demo/mxnet/mxnet_demo.py

@@ -0,0 +1,93 @@
+import mxnet as mx
+
+# Here we import LogWriter so that we can write log data while MXNet is training
+from visualdl import LogWriter
+
+# Download MNIST data
+mnist = mx.test_utils.get_mnist()
+batch_size = 100
+
+
+# Provide a folder to store data for log, model, image, etc. VisualDL's visualization will be
+# based on this folder.
+logdir = "./tmp"
+
+# Initialize a logger instance. Parameter 'sync_cycle' means write a log every 10 operations on
+# memory.
+logger = LogWriter(logdir, sync_cycle=10)
+
+# mark the components with 'train' label.
+with logger.mode("train"):
+    # scalar0 is used to record scalar metrics while MXNet is training. We will record accuracy.
+    # In the visualization, we can see the accuracy is increasing as more training steps happen.
+    scalar0 = logger.scalar("scalars/scalar0")
+
+# Record training steps
+cnt_step = 0
+
+
+# MXNet provides many callback interface. Here we define our own callback method and it is called
+# after every batch.
+# https://mxnet.incubator.apache.org/api/python/callback/callback.html
+def add_scalar():
+    def _callback(param):
+        with logger.mode("train"):
+            global cnt_step
+            # Here the value is the accuracy we want to record
+            # https://mxnet.incubator.apache.org/_modules/mxnet/callback.html
+            name_value = param.eval_metric.get_name_value()
+            for name, value in name_value:
+                scalar0.add_record(cnt_step, value)
+                cnt_step += 1
+    return _callback
+
+
+# Start to build CNN in MXNet, train MNIST dataset. For more info, check MXNet's official website:
+# https://mxnet.incubator.apache.org/tutorials/python/mnist.html
+
+import logging
+logging.getLogger().setLevel(logging.DEBUG)  # logging to stdout
+
+train_iter = mx.io.NDArrayIter(mnist['train_data'], mnist['train_label'], batch_size, shuffle=True)
+val_iter = mx.io.NDArrayIter(mnist['test_data'], mnist['test_label'], batch_size)
+
+data = mx.sym.var('data')
+# first conv layer
+conv1 = mx.sym.Convolution(data=data, kernel=(5, 5), num_filter=20)
+tanh1 = mx.sym.Activation(data=conv1, act_type="tanh")
+pool1 = mx.sym.Pooling(data=tanh1, pool_type="max", kernel=(2, 2), stride=(2, 2))
+# second conv layer
+conv2 = mx.sym.Convolution(data=pool1, kernel=(5, 5), num_filter=50)
+tanh2 = mx.sym.Activation(data=conv2, act_type="tanh")
+pool2 = mx.sym.Pooling(data=tanh2, pool_type="max", kernel=(2, 2), stride=(2, 2))
+# first fullc layer
+flatten = mx.sym.flatten(data=pool2)
+fc1 = mx.symbol.FullyConnected(data=flatten, num_hidden=500)
+tanh3 = mx.sym.Activation(data=fc1, act_type="tanh")
+# second fullc
+fc2 = mx.sym.FullyConnected(data=tanh3, num_hidden=10)
+# softmax loss
+lenet = mx.sym.SoftmaxOutput(data=fc2, name='softmax')
+
+# create a trainable module on CPU
+lenet_model = mx.mod.Module(symbol=lenet, context=mx.cpu())
+
+
+# train with the same
+lenet_model.fit(train_iter,
+                eval_data=val_iter,
+                optimizer='sgd',
+                optimizer_params={'learning_rate': 0.1},
+                eval_metric='acc',
+                # integrate our customized callback method
+                batch_end_callback=[add_scalar()],
+                num_epoch=2)
+
+test_iter = mx.io.NDArrayIter(mnist['test_data'], None, batch_size)
+prob = lenet_model.predict(test_iter)
+test_iter = mx.io.NDArrayIter(mnist['test_data'], mnist['test_label'], batch_size)
+
+# predict accuracy for lenet
+acc = mx.metric.Accuracy()
+lenet_model.score(test_iter, acc)
+print(acc)