Ubuntu install puppeteer

Installing puppeteer into a fresh AWS EC2 Ubuntu 18.04 and its dependencies.

sudo apt update
sudo apt upgrade
sudo apt install npm
npm i puppeteer

Now run a simple puppeteer script and you will encounter errors about missing shared libraries. A simple script below, lets name it simple.js

const puppeteer = require('puppeteer');
async function run () {
   const browser = await puppeteer.launch({
      defaultViewport: {width: 1920, height: 1080}
   });
   const page = await browser.newPage();
   await page.goto('https://www.google.com/search?q=bitcoin');
   await page.screenshot({path: 'screenshot.png'});
   browser.close();
}
run(); 
node simple.js

Now install the dependencies and rerun.

sudo apt install libx11-xcb1 libxcomposite1 libxcursor1 libxdamage1 libxi-dev libxtst-dev libnss3 libcups2 libxss1 libxrandr2 libasound2 libatk1.0-0 libatk-bridge2.0-0 libpangocairo-1.0-0 libgtk-3-0

Ubuntu install squid

Setting up undetectable proxy in a fresh new Ubuntu 18.04 in AWS EC2

sudo apt update
sudo apt upgrade
sudo apt install squid

Edit the configuration file at /etc/squid/squid.conf . Change the settings to the following below

http_access allow all
forwarded_for delete
via off
request_header_access Cache-Control deny all 

I set http_access to allow all because I’m using AWS security group to white list IP and port to my proxy server. After making changes to the settings, restart squid.

sudo systemctl restart squid

Ubuntu install golang

As usual, before installing anything in Ubuntu, run

sudo apt update
sudo apt upgrade

Once done, you can start installing golang. The easiest method to install using the below command.

sudo apt install golang-go

Next is to test if you had successfully install golang. Create a hello.go file, paste the contents below into the hello.go file and save it.

package main

import "fmt"

func main() {
    fmt.Println("hello world")
}

To execute the file, run the command below.

go run hello.go

Ubuntu install sysstat

Ubuntu install sysstat

sudo apt install sysstat

Once installation completed, enable sysstat by setting the value to true.

sudo vim /etc/default/sysstat

ubuntu-install-sysstat-01

After that restart the sysstat service

sudo service sysstat restart

By default sysstat sample the CPU usage every 10 minutes at 5, 15, 25, 35, 45 and 55 minutes of the hour. To modify it,

sudo vim /etc/cron.d/sysstat

ubuntu-install-sysstat-02

Viewing the stats

To view latest stats

sar

To view yesterday’s stats

sar -f /var/log/sysstat/sa$(date +%d -d 'yesterday')

To view specific date stats

sar -f /var/log/sysstat/sa$(date +%d -d '2019-04-27')

 

Ubuntu install Plex media server

Ubuntu install Plex media server

Go to Plex website to get the download URL

ubuntu-install-plex-media-server

cd /tmp
wget https://downloads.plex.tv/plex-media-server/1.14.1.5488-cc260c476/plexmediaserver_1.14.1.5488-cc260c476_amd64.deb
sudo dpkg -i plexmediaserver_1.14.1.5488-cc260c476_amd64.deb
systemctl status plexmediaserver

ubuntu-install-plex-media-server

Go to http://127.0.0.1:32400/web and start configuring your Plex

Assuming that the videos are in your external hard disk connected to your Ubuntu desktop. You’ll need to run these commands to give user plex to access your drive.

MYGROUP="$USER"
sudo usermod -a -G $MYGROUP plex
sudo chown $USER:$MYGROUP /media/$USER
sudo chmod 750 /media/$USER
sudo setfacl -m g:$MYGROUP:rwx /media/$USER
sudo service plexmediaserver restart

to check your group or the group user plex is in

groups
groups plex

 

Ubuntu disable sleep on lid close

Ubuntu disable sleep on lid close

I prefer to install Ubuntu Desktop version instead of Ubuntu Server version because it takes care of all the drivers for my laptop. After the installation I’ll disable the GUI.

The screen will stay on at the login prompt and since it is a laptop, there isn’t a power off for the monitor only. Closing the lid will cause the laptop to sleep.

To prevent it from going to sleep. Do these.

ubuntu-disable-sleep-on-lid-close

Uncomment the 2 lines below and set it to ignore.

ubuntu-disable-sleep-on-lid-close

Lastly restart the service.

sudo service systemd-logind restart

 

Ubuntu 18.04 install MySQL Server

Ubuntu 18.04 install MySQL Server

Installation for MySQL is slightly different in 18.04 as compared to 16.04.

sudo apt update
sudo apt install mysql-server -y
sudo mysql -uroot

Now in MySQL console, type the following to check the users.

select user,host,plugin from mysql.user;
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'NEWPASSWORD';
update mysql.user set host='%' where user='root';
exit;

Comment the line bind-address if you like to listen on all addresses.

sudo service mysql restart

 

Ubuntu install Selenium

Ubuntu install Selenium

Most sites can be crawled with simple wget in Bash but some are loaded at runtime using Javascripts. These sites cannot be crawled using simple wget in Bash. You’ll need something like Selenium.

To install Selenium, Firefox and xvfb. Run these commands.

sudo apt update
sudo apt install python-pip firefox xvfb -y
echo export DISPLAY=:99|sudo tee -a /etc/environment
pip install selenium

Download geckodriver. Extract it and move to /usr/bin/

wget https://github.com/mozilla/geckodriver/releases/download/v0.24.0/geckodriver-v0.24.0-linux64.tar.gz
tar -xzvf geckodriver-v0.24.0-linux64.tar.gz
sudo mv geckodriver /usr/bin/

Login as root, and set to run xvfb at reboot.

sudo su
crontab -e
@reboot Xvfb :99 -ac
reboot now

Create a simple python script to test if your installation is successful.

vim test.py
from selenium import webdriver
driver = webdriver.Firefox()
try:
  driver.get('https://www.bing.com')
  print driver.title
finally:
  driver.quit()
python test.py