Deploy Django 4.0 on Shared Hosting

Create a Python App

Log in to CPanel and open Setup Python App.

Click on Create Application

  • Select Python Version 3.9.12
  • Application Root is the directory where you will place the code files of your Django project. Make sure that it is a different folder than your domain root.
  • Application URL is the domain or subdomain where your app will run
  • In the Application Startup File, type passenger_wsgi.py
  • In the Application Entry Point, type application

After setting all these, click on the Create button.

Setup the Database

Open MySQL Databases in Cpanel

Create a new Database and note the database name. We will need it later.

Create a new MySQL user and note the username and password. We will need it later.

Add the new user to the new Database

Grant all the permissions to the user and select Make Changes

Upload your project

Open the File Manager and go to the Application root you specified in the part 1

Zip your project. Upload it to this folder and extract the zip file. Your project files should be in the same folder as the passenger_wsgi.py file. Make sure that manage.py and passenger_wsgi.py are in the same folder.

Edit the passenger_wsgi.py file.

Delete everything from this file and add the following code:

from base.wsgi import applicationCopy

Where base is your project folder. It is the same folder that contains your settings.py file. It will be different if you are not using the test project that I provided. You can locate your wsgi.py file and import application from it.

Now edit your settings.py

Add your domain to the ALLOWED_HOSTS list. If there is a www version of your domain, add that too. Do not use http:// to https://ALLOWED_HOSTS = ['django.umer.link']Copy

In the DATABASES dictionary, modify the default database. 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'databasename', 'USER': 'databaseusername', 'PASSWORD': 'databasepassword', 'HOST': 'localhost', 'PORT': '3306', }Copy

Make sure to replace databasename with the database name, databaseusername with the database username and databasepassword with the database password.

Now go to the end of the file, modify STATIC_URL and add the STATIC_ROOTSTATIC_URL = '/static/' STATIC_ROOT = '/home/username/domainroot/static'Copy

Replace the username with your CPanel username. The domainroot will be public_html if you are deploying on the main domain of your hosting account. If you are deploying on a subdomain or an add on domain, it will be name of the addon domain or the subdomain.

Now edit the __init__.py file

Add the following codeimport pymysql pymysql.install_as_MySQLdb()Copy

Complete configuration from the Terminal

Open the Terminal in your CPanel. If you can not find the terminal, go to Manage Shell and Enable SSH Access. You can also do these steps using SSH but using the terminal is easy.

Copy the command to enter the virtual environment from your python app.

Run the copied command in the terminal and press enter to enter the virtual environment,

Install Django 4.0.4 by running the following command.pip install django==4.0.4Copy

pymysql is required for using the MySQL database. Install it using pip. Here you will install any other modules required by your Django app.pip install pymysqlCopy

If your migrations are not complete, then make migrations.python manage.py makemigrationsCopy

Run migrate to create tables in the database.python manage.py migrateCopy

Run collectstatic to move the static files to the static root folder specified in settings.python manage.py collectstaticCopy

Run createsuperuser to ass a user to the Django admin panelpython manage.py createsuperuserCopy

Finally, restart the python app.

Your Django app is deployed successfully. Open the url of your app to see if it is working fine. If you see the phusion passenger error page, you can find the error in the stderr.log file for debugging.

Reference: https://pythonfusion.com/deploy-django-on-shared-hosting/

Get user token and user info from IdentityServer4

IdentityServer4 is an OpenID Connect and OAuth 2.0 framework for ASP.NET Core.

Use the PasswordTokenRequest and RequestPasswordTokenAsync to get the access token. replace clientId and secret with the values from your Identity Server, then use the UserInfoRequest and pass your access token to GetUserInfoAsync to get the user claims:

public class TokenService
{
    private DiscoveryDocumentResponse _discDocument { get; set; }
    public TokenService()
    {
        using (var client = new HttpClient())
        {
            _discDocument = client.GetDiscoveryDocumentAsync("http://localhost:5000/.well-known/openid-configuration").Result;
        }
    }
    public async Task<TokenResponse> GetToken(string userName, string password)
    {
        using (var client = new HttpClient())
        {
            var tokenResponse = await client.RequestPasswordTokenAsync(new PasswordTokenRequest
            {
                Address = _discDocument.TokenEndpoint,
                ClientId = "clientId",
                ClientSecret = "secret",
                Scope = "openid",
                GrantType = "password",
                UserName = userName,
                Password = password
            });

            if (tokenResponse.IsError)
            {
                throw new Exception("Token Error");
            }
            return tokenResponse;
        }
    }

    public async Task<string> GetUserInfo(string accessToken)
    {
        using (var client = new HttpClient())
        {
            var response = await client.GetUserInfoAsync(new UserInfoRequest()
            {
                Address = _discDocument.UserInfoEndpoint,
                Token = accessToken
            });

            if (response.IsError)
            {
                throw new Exception("Invalid username or password");
            }
            return response.Raw;
        }
    }
}

New Raspberry Pi OS released

New Raspberry Pi OS Bullseye is released. It comes with lots of new features.

  * Based on Debian version 11 (bullseye)
  * Desktop components (lxpanel and all plugins, libfm, pcmanfm) now built against GTK+3
  * Applications (piwiz, pipanel, rc_gui, lxinput) now built against GTK+3
  * PiXflat GTK+3 theme updated with numerous changes to support the above
  * GTK+3 : toolbar icon size setting added
  * GTK+3 : ability to request client-side decoration on windows added
  * GTK+3 : setting for indent for frame labels in custom style added
  * mutter window manager used instead of openbox on devices with 2GB or more of RAM
  * mutter : title bar icon behaviour and appearance modified to match openbox
  * mutter : additional keyboard shortcuts added
  * mutter : various performance enhancements
  * mutter compatibility added to screen magnifier
  * Numerous changes to Appearance Settings application to support GTK+3 and mutter
  * Updater plugin added to lxpanel to detect and install software updates
  * File manager view options simplified to either list or icons, with separate menu option for thumbnails
  * New file manager toolbar icons
  * KMS used as default display driver
  * Modifications to HDMI audio output selection to support the above
  * xcompmgr enabled when openbox is running under KMS
  * New default camera subsystem based on libcamera
  * New camera demo applications (libcamera-still and libcamera-vid) have replaced raspistill and raspivid
  * Legacy camera subsystem removed from 64-bit RPi OS (still available on 32-bit)
  * Chromium upgraded to version 92.0.4515.98
  * VLC media player upgraded to version 3.0.16
  * Spurious drive removal warning after use of SD card copier removed
  * Bookshelf application now includes Custom PC magazine
  * Various translation updates - Italian, Korean, Polish, German, Armenian
  * Startup wizard now installs Japanese fonts if needed
  * Progress and information dialog boxes for lxpanel plugins now common to lxpanel, rather than in individual plugins
  * Icon handling code for lxpanel plugins now common to lxpanel
  * Package with 4K version of Raspberry Pi wallpaper added to Recommended Software
  * Python Games and Minecraft removed from Recommended Software - neither is compatible with bullseye
  * Bluetooth pairing and connection dialogs updated for compatibility with more devices
  * Bluetooth devices always disconnected before removal to speed up removal process
  * Bluetooth pairing dialog now only shows devices which offer services which are usable by Pi
  * Separate Bluetooth unpair dialog removed - unpair now an option for each individual device
  * Bug fix - mutter : header bar colours not updating when theme is changed
  * Bug fix - GTK+3 : tooltips being displayed incorrectly at bottom of screen
  * Bug fix - lxpanel : crash when using keyboard shortcut to enable magnifier when magnifier not installed
  * Bug fix - lxpanel : lockup in Bluetooth plugin when connecting to certain devices
  * Bug fix - lxpanel : discoverable mode icon could get out of sync with underlying Bluetooth system state
  * Bug fix - piwiz : missing cities in timezone list
  * Bug fix - piwiz : country-specific language packages not being installed
  * Bug fix - bookshelf : now waits for longer between packets before timing out
  * Bug fix - accented characters now displayed correctly in localisation dialogs
  * Raspberry Pi firmware e2bab29767e51c683a312df20014e3277275b8a6
  * Linux kernel 5.10.63

https://www.raspberrypi.com/software/operating-systems/

Use SharpPcap to construct and send UDP packet

SharpPcap is fully managed, cross platform (Windows, Mac, Linux) .NET library for capturing packets from live and file based devices. Now we want to send UPD packet using SharpPcap.

public int SendUdp(byte[] dgram, int bytes, IPEndPoint endPoint)
{
    //construct ethernet packet
    var ethernet = new EthernetPacket(PhysicalAddress.Parse("112233445566"), PhysicalAddress.Parse("665544332211"), EthernetType.IPv4);
    //construct local IPV4 packet
    var ipv4 = new IPv4Packet(IPAddress.Parse("192.168.0.4"), endPoint.Address);
    ethernet.PayloadPacket = ipv4;
    //construct UDP packet
    var udp = new UdpPacket(12345, (ushort)endPoint.Port);
    //add data in
    udp.PayloadData = dgram;
    ipv4.PayloadPacket = udp;

    _device.SendPacket(ethernet);
    return bytes;
}

SharpPcap can be found https://github.com/chmorgan/sharppcap.

How to change Raspberry Pi screen resolution in headless mode

My Raspberry Pi screen resolution being stuck at 800×600 with VNC access to a headless Raspberry PI with Raspberry Pi OS. I found that if I comment out the lines towards the end (as shown below) I was able to achieve a 1920×1080 resolution with my remote headless connection.

[pi4]
# Enable DRM VC4 V3D driver on top of the dispmanx display stack
#dtoverlay=vc4-fkms-v3d
#max_framebuffers=2

[all]
#dtoverlay=vc4-fkms-v3d

How to use SharpExt4 to access Raspberry Pi SD Card Linux partition

In my previous post, I introduced the SharpExt4 .Net library. In this post, I will show how to use SharpExt4 to access the Raspberry Pi SD card from Windows OS.

  • Take out the Raspberry Pi SD card and insert it into a USB card reader
  • Run “diskpart” from Windows command prompt and find out the SD card disk number and partition number. In my case, the disk number is 3 and the partition is 2.
  • Clone the SharpExt4 from GitHub, and open Visual Studio as Admin

Note: If you want to access physical drive, you must run application in admin permission

  • Open SharpExt4 and edit the Program.cs from the Sample project
static void Main(string[] args)
{
    //Open Raspberry Pi SD card, see diskpart disk number
    var disk = ExtDisk.Open(3);
    //Get the file system, see diskpart partition number
    var fs = ExtFileSystem.Open(disk.Parititions[1]);
    //List all directories in root folder
    foreach (var file in fs.GetDirectories("/", "*", SearchOption.TopDirectoryOnly))
    {
        Console.WriteLine(file);
    }
}

Run the Sample project and see the result to list all the folders in Raspberry Pi root folder.

SharpExt4, a .Net library, provides read/write Linux ext2/3/4 file system from Windows application (continue)

Follow my previous post: https://www.nickdu.com/?p=896

Open Ext4 file system.

 //Get the file system
 var fs = ExtFileSystem.Open(disk.Parititions[0]);

Sample code to open a file for read

//Open a file for read
var file = fs.OpenFile("/etc/shells", FileMode.Open, FileAccess.Read);
//Check the file length
var filelen = file.Length;
var buf = new byte[filelen];
//Read the file content
var count = file.Read(buf, 0, (int)filelen);
file.Close();
var content = Encoding.Default.GetString(buf);
Console.WriteLine(content);

Sample code for listing all files in a folder

//List all files in /etc folder
foreach (var file in fs.GetFiles("/etc", "*", SearchOption.AllDirectories))
{
    Console.WriteLine(file);
}

Sample code for file creation

//Open a file for write
var file = fs.OpenFile("/etc/test", FileMode.Create, FileAccess.Write);
var hello = "Hello World";
var buf = Encoding.ASCII.GetBytes(hello);
//Write to file
file.Write(buf, 0, buf.Length);
file.Close();

Full ExtDisk APIs

public sealed class ExtDisk : IDisposable
{
    public IList<Partition> Parititions { get; }
    public Geometry Geometry { get; }
    public ulong Capacity { get; }

    public static ExtDisk Open(string imagePath);
    public static ExtDisk Open(int DiskNumber);
    public sealed override void Dispose();
    public byte[] GetMasterBootRecord();

}

Full ExtFileSystem APIs

public sealed class ExtFileSystem : IDisposable
{
    public static string MountPoint { get; }
    public string Name { get; }
    public string VolumeLabel { get; }
    public bool CanWrite { get; }
    public string Description { get; }

    public static ExtFileSystem Open(Partition partition);
    public void CopyFile(string sourceFile, string destinationFile, bool overwrite);
    public void CreateDirectory(string path);
    public void CreateHardLink(string target, string path);
    public void CreateSymLink(string target, string path);
    public void DeleteDirectory(string path);
    public void DeleteFile(string path);
    public bool DirectoryExists(string path);
    public sealed override void Dispose();
    public bool FileExists(string path);
    public ValueType GetCreationTime(string path);
    public string[] GetDirectories(string path, string searchPattern, SearchOption searchOption);
    public ulong GetFileLength(string path);
    public string[] GetFiles(string path, string searchPattern, SearchOption searchOption);
    public DateTime GetLastAccessTime(string path);
    public DateTime GetLastWriteTime(string path);
    public uint GetMode(string path);
    public Tuple<uint, uint> GetOwner(string path);
    public void MoveDirectory(string sourceDirectoryName, string destinationDirectoryName);
    public ExtFileStream OpenFile(string path, FileMode mode, FileAccess access);
    public string ReadSymLink(string path);
    public void RenameFile(string sourceFileName, string destFileName);
    public void SetCreationTime(string path, DateTime newTime);
    public void SetLastAccessTime(string path, DateTime newTime);
    public void SetLastWriteTime(string path, DateTime newTime);
    public void SetMode(string path, uint mode);
    public void SetOwner(string path, uint uid, uint gid);
    public sealed override string ToString();
    public void Truncate(string path, ulong size);
}

Full ExtFileStream APIs

public class ExtFileStream : Stream
{
    public ExtFileStream(ExtFileSystem fs, string path, FileMode mode, FileAccess access);

    public override long Position { get; set; }
    public override long Length { get; }
    public override bool CanWrite { get; }
    public override bool CanRead { get; }
    public override bool CanSeek { get; }

    public override void Close();
    public override void Flush();
    public override int Read(byte[] array, int offset, int count);
    public override long Seek(long offset, SeekOrigin origin);
    public override void SetLength(long value);
    public override void Write(byte[] array, int offset, int count);

}

The full SharpExt4 library can be found at my GitHub.

https://github.com/nickdu088/SharpExt4

SharpExt4, a .Net library, provides read/write Linux ext2/3/4 file system from Windows application (continue)

Follow my last post SharpExt4, a .Net library, provides read/write Linux ext2/3/4 file system from Windows application.

The lwext4 is a great start point for me (thanks the author of lwext4), and it provides the core implementation of Linux ext2/3/4 filesystem.

What I need to do:

  1. To port the entire lwext4 project over to Visual Studio C/C++ (MSVC) environment, and compile it as a static library.
  2. To create a clr wrapper around the lwext4 static library and compiled as a .Net assembly DLL.
  3. To provide a .Net friendly interface for .Net Application to use.

When creating this .Net library, I would like to access not only physical Linux disk directly, but also Linux disk image file.

SharpExt4 provides two open disk APIs:

//Open physical Linux disk
ExtDisk SharpExt4.ExtDisk.Open(int DiskNumber);
//Open Linux disk image file
ExtDisk SharpExt4.ExtDisk.Open(String imagePath);

Open Linux disk image allows developer to directly manipulate the saved Linux disk image file, e.g. Raspberry Pi OS image or Debian OS image. The saved Linux disk image must be raw format. This API doesn’t support Virtual Machine disk files (VHD, VDI, XVA, VMDK, etc).

This API also support open USB/Hard drive as physical disk by giving disk number. If you have a USB disk or hard drive formatted as ext2/3/4 file system, this API allows the developer to read/write directly.

To be continue…

Next post: https://www.nickdu.com/?p=912

SharpExt4, a .Net library, provides read/write Linux ext2/3/4 file system from Windows application

As a day to day Windows user, it’s not easy to access Linux file system. Windows doesn’t natively supports Linux Extended file system access.

I have been working on Linux ARM IoT device, and it’s so annoying to be back and forth between Windows working PC and Linux development PC. I regularly need to burn/flash SD card, read/write file, remove/open directory in Linux device image.

These are the findings so far:

  1. DiscUtils, is a .NET library to read and write ISO files and Virtual Machine disk files (VHD, VDI, XVA, VMDK, etc). DiscUtils also provides limited access to ext2/3/4 filesystem.
  2. Ext2Fsd is another Windows file system driver for the Ext2, Ext3, and Ext4 file systems. It allows Windows to read Linux file systems natively, providing access to the file system via a drive letter that any program can access. But it stops developing since 2017.
  3. DiskInternals Linux Reader is a freeware application from DiskInternals, developers of data recovery software. But it doesn’t have API for .Net Framework.
  4. Ext2explore is an open-source application that works similarly to DiskInternals Linux Reader—but only for Ext4, Ext3, and Ext2 partitions. It stops developing for a long time and is read only.

I decided to implement my own library. Finally, I found this, lwext4, a C library to provide ext2/3/4 filesystem for microcontrollers. According to the author, the library has some cool and unique features in microcontrollers world:

  • directory indexing – fast file find and list operations
  • extents – fast big file truncate
  • journaling transactions & recovery – power loss resistance

Lwext4 is an excellent choice for SD/MMC card, USB flash drive or any other wear leveled memory types. However it is not good for raw flash devices.

To be continue.

Next post https://www.nickdu.com/?p=896