Машинное обучение и Большие данные: различия между версиями
Перейти к навигации
Перейти к поиску
Artem (обсуждение | вклад) |
Artem (обсуждение | вклад) |
||
| Строка 31: | Строка 31: | ||
* [https://developer.nvidia.com/rdp/cudnn-download cuDNN SDK] | * [https://developer.nvidia.com/rdp/cudnn-download cuDNN SDK] | ||
* [https://developer.nvidia.com/nvidia-tensorrt-7x-download TensorRT] | * [https://developer.nvidia.com/nvidia-tensorrt-7x-download TensorRT] | ||
* [https://towardsdatascience.com/installing-tensorflow-gpu-in-ubuntu-20-04-4ee3ca4cb75d] | |||
= Курсы = | = Курсы = | ||
Версия от 02:10, 28 марта 2021
Hardware
- Lenovo x230 + eGPU
- Expresscard V8.0 EXP GDC Beast PCIe PCI-E
- Блок питания на 350-600 ватт
- Nvidia GeForce 760 4gb
- Similar setup
- Troubleshooting
- 16 GB of mem will produce lags. Remove 1 stick of mem to 8 GB
- Be sure that you GPU conneted to power fully(8+6 or 8+8) - it can produce 43 error
- eGPU setup Lenovo Thinkpad x230 with GTX 760 Part 1 ( setup )
- eGPU setup Lenovo Thinkpad x230 with GTX 760 Part 2 ( Fixing Error 12)
- eGPU setup Lenovo Thinkpad x230 with GTX 760 Part 3 ( Gameplay )
- In Windows go to Control panel, Hardware setup, Nvidia settings, 3d graphics, There you can select default video adapter
- But it won't help to use GPU in games - you need to connect external screen to GPU and disable laptop screen. Then games will run on eGPU.
Software
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600 wget https://developer.download.nvidia.com/compute/cuda/11.2.2/local_installers/cuda-repo-ubuntu2004-11-2-local_11.2.2-460.32.03-1_amd64.deb sudo dpkg -i cuda-repo-ubuntu2004-11-2-local_11.2.2-460.32.03-1_amd64.deb sudo apt-key add /var/cuda-repo-ubuntu2004-11-2-local/7fa2af80.pub sudo apt-get update sudo apt-get -y install cuda
Курсы
- https://medium.com/nuances-of-programming/%D1%82%D0%BE%D0%BF-10-%D0%BA%D1%83%D1%80%D1%81%D0%BE%D0%B2-%D0%BF%D0%BE-%D0%BC%D0%B0%D1%88%D0%B8%D0%BD%D0%BD%D0%BE%D0%BC%D1%83-%D0%B8-%D0%B3%D0%BB%D1%83%D0%B1%D0%BE%D0%BA%D0%BE%D0%BC%D1%83-%D0%BE%D0%B1%D1%83%D1%87%D0%B5%D0%BD%D0%B8%D1%8E-%D0%B2-2020-1e1d870a24b7
- https://skillbox.ru/course/profession-data-scientist/
- https://new.geekbrains.ru/machine-learning
- https://praktikum.yandex.ru/data-scientist/
- https://www.coursera.org/learn/machine-learning
- https://www.coursera.org/specializations/deep-learning
- https://ru.coursera.org/specializations/machine-learning-data-analysis
- https://www.udacity.com/course/machine-learning--ud262
- https://www.udacity.com/course/intro-to-machine-learning--ud120
Большие данные
- Лучшие в своём деле: Артур Хачуян | Большие данные — Big Data | ЛСД #7 https://www.youtube.com/watch?v=frLydE1UCvA
- «На чём корпорации вертели вашу приватность», Артур Хачуян (Tazeros Global) https://www.youtube.com/watch?v=8IJxO44kq24
Методы
- Теорема Байеса
- Функции ошибки и регуляризация
- Расстояние Кульбака-Лейблера и перекрестная энтропия
- Градиентный спуск: основы
- Граф вычислений и дифференцирование на нем
- Перцептрон
- Глубокие нейронные сети
- Классификация
- Кластеризация
- Регрессия
- Машинное зрение
- Метод к-средних
- word2vec
Библиотеки
- NumPy - https://www.numpy.org/
- SciPy - https://www.scipy.org/
- Pandas - https://pandas.pydata.org/
- Scikit-learn - https://scikit-learn.org/stable/
- Matplotlib - https://matplotlib.org/
- Gensim - https://radimrehurek.com/gensim/
- H2O - https://github.com/h2oai/h2o-tutorials
- XGBoost - https://xgboost.readthedocs.io/en/latest/
- CatBoost - https://yandex.ru/dev/catboost/
- Theano - http://deeplearning.net/software/theano/
- TensorFlow - https://www.tensorflow.org/
- sknn - https://scikit-neuralnetwork.readthedocs.io/en/latest/
- Theanets - https://github.com/lmjohns3/theanets
- Keras - https://keras.io/
- OpenCV - https://opencv.org/
Датасеты
- http://yann.lecun.com/exdb/mnist/
- http://www.image-net.org/
- https://en.wikipedia.org/wiki/List_of_datasets_for_machine_learning_research
- https://storage.googleapis.com/openimages/web/index.html
- https://github.com/philipperemy/timit
- https://www.nist.gov/programs-projects/face-recognition-grand-challenge-frgc
Железо и драйверы
- https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal
- https://www.tensorflow.org/install/gpu
- https://cloud.google.com/tpu/
Темы
- https://en.wikipedia.org/wiki/Outline_of_artificial_intelligence#Branches_of_artificial_intelligence
Face Recognition
Speech Recognition
Image Object Recognition
Anomaly Detection
Prediction
StereoVision
Некоторые полезные ресурсы
- https://ods.ai/
- https://en.wikipedia.org/wiki/Outline_of_artificial_intelligence
- https://en.wikipedia.org/wiki/List_of_artificial_intelligence_projects
- https://en.wikipedia.org/wiki/List_of_datasets_for_machine_learning_research
- http://wordnet.princeton.edu/
- http://project.phil.spbu.ru/RussNet/index_ru.shtml
- http://wordnet.ru/
- http://www.vision.caltech.edu/Image_Datasets/Caltech101/
- https://www.scipy.org/
- http://www.numpy.org/
- http://leenissen.dk/fann/html/files/fann_train-h.html
- https://archive.ics.uci.edu/ml/datasets/mushroom
- https://archive.ics.uci.edu/ml/index.php
- https://www.kaggle.com/
- https://keras.io/
- https://xgboost.readthedocs.io/en/latest/
- https://lasagne.readthedocs.io/en/latest/
- http://deeplearning.net/software/theano/index.html
- https://catboost.yandex/
- https://arxiv.org/
- https://arxiv.org/list/cs.CV/recent
- https://yandex.ru/dev/catboost/