How to List all Files and Directories in FTP Server using Python

Listing all files and directories in an FTP server using LIST, NLST and MLSD FTP commands in Python using the built-in ftplib module.
Abdou Rockikz · 6 min read · Updated mar 2020 · Python Standard Library


FTP (File Transfer Protocol) is the standard network protocol that enables computers to transfer files across the network. In this tutorial, you will learn how you can connect to a FTP server and list all files and directories on it, you will also get familiar with Python's built-in ftplib module and some of its methods.

ftplib comes pre-installed with Python, so if you have Python installed on your machine, you're good to go. Open up a new Python file and follow along, let's import the necessary module for the tutorial:

import ftplib
import os
from datetime import datetime

In a regular FTP server, you need credentials (username and password) in order to properly login into one, but for tutorial, we gonna use the FTP server of University of Edinburgh which enables users to login anonymously:

FTP_HOST = "ftp.ed.ac.uk"
FTP_USER = "anonymous"
FTP_PASS = ""

Below are utility functions that will help us later print our list of files and directories:

# some utility functions that we gonna need
def get_size_format(n, suffix="B"):
    # converts bytes to scaled format (e.g KB, MB, etc.)
    for unit in ["", "K", "M", "G", "T", "P"]:
        if n < 1024:
            return f"{n:.2f}{unit}{suffix}"
        n /= 1024

def get_datetime_format(date_time):
    # convert to datetime object
    date_time = datetime.strptime(date_time, "%Y%m%d%H%M%S")
    # convert to human readable date time string
    return date_time.strftime("%Y/%m/%d %H:%M:%S")

Now, let's connect to our server using FTP client class:

# initialize FTP session
ftp = ftplib.FTP(FTP_HOST, FTP_USER, FTP_PASS)

When writing the code of this tutorial, I've encountered some problems working with non latin characters, as Python uses ISO-8859-1 as default encoding, as a result, let's change the encoding to UTF-8:

# force UTF-8 encoding
ftp.encoding = "utf-8"

Now that we are inside the server, let's print the welcome message that is sent by the server once we're connected:

# print the welcome message
print(ftp.getwelcome())

Here is the output for this server:

         Welcome to the University of Edinburgh Anonymous FTP server        
 ===========================================================================

  When requested for a username enter 'ftp' or 'anonymous'.  If you have    
  problems, try using a dash (-) as the first character of your password.   
  If you still have problems or wish to make a comment then send email to   
  ftpmaster@ed.ac.uk.

  All transfers are logged.
220 FTP Server

Let's start calling some commands, the first method we gonna use is cwd(), which changes the current working directory, since we're in the root directory, let's change to some directory that has some files inside it:

# change the current working directory to 'pub' folder and 'maps' subfolder
ftp.cwd("pub/maps")

Listing files and directories:

# LIST a directory
print("*"*50, "LIST", "*"*50)
ftp.dir()

Here is a part of the output:

************************************************** LIST **************************************************
-rw-r--r--   1 1407     bin         25175 Jul  8  1991 JIPS-map.910704-1.ps.Z
-rw-r--r--   1 1407     bin         25714 Jul 30  1991 JIPS-map.910730-1.ps.Z
-rw-r--r--   1 1407     bin         25980 Aug  2  1991 JIPS-map.910802-1.ps.Z
-rw-r--r--   1 1407     bin         26812 Aug  7  1991 JIPS-map.910806-1.ps.Z
-rw-r--r--   1 1407     bin         26673 Oct 11  1991 JIPS-map.911011-1.ps.Z
...<SNIPPED>...

Quite similar to the output provided by the ls command. However, this uses FTP's LIST command which is obsolete by now. Also, as you may already noticed, it does not return any value, it just prints to the screen the directories and files in the current working directory.

Another alternative is to use NLST command:

# NLST command
print("*"*50, "NLST", "*"*50)
print("{:20} {}".format("File Name", "File Size"))
for file_name in ftp.nlst():
    file_size = "N/A"
    try:
        ftp.cwd(file_name)
    except Exception as e:
        ftp.voidcmd("TYPE I")
        file_size = get_size_format(ftp.size(file_name))
    print(f"{file_name:20} {file_size}")

Output:

************************************************** NLST **************************************************
File Name            File Size
backbone.t3-ps.Z     23.39KB
backbone.t1t3-ps.Z   24.56KB
ripe-map06-netnums.ps.Z 29.54KB
edlana4bw.ps.Z       63.34KB
...<SNIPPED>...

But, as you may see, NLST command returns only the names of files and directories, nothing else, we want something that provides us the list of names as well as their meta data such as permissions, size, date of last modification, etc.

Here we use MLSD command that comes into the rescue:

print("*"*50, "MLSD", "*"*50)
# using the MLSD command
print("{:30} {:19} {:6} {:5} {:4} {:4} {:4} {}".format("File Name", "Last Modified", "Size",
                                                    "Perm","Type", "GRP", "MODE", "OWNER"))
for file_data in ftp.mlsd():
    # extract returning data
    file_name, meta = file_data
    # i.e directory, file or link, etc
    file_type = meta.get("type")
    if file_type == "file":
        # if it is a file, change type of transfer data to IMAGE/binary
        ftp.voidcmd("TYPE I")
        # get the file size in bytes
        file_size = ftp.size(file_name)
        # convert it to human readable format (i.e in 'KB', 'MB', etc)
        file_size = get_size_format(file_size)
    else:
        # not a file, may be a directory or other types
        file_size = "N/A"
    # date of last modification of the file
    last_modified = get_datetime_format(meta.get("modify"))
    # file permissions
    permission = meta.get("perm")
    
    # get the file unique id
    unique_id = meta.get("unique")
    # user group
    unix_group = meta.get("unix.group")
    # file mode, unix permissions 
    unix_mode = meta.get("unix.mode")
    # owner of the file
    unix_owner = meta.get("unix.owner")
    # print all
    print(f"{file_name:30} {last_modified} {file_size:7} {permission:5} {file_type:4} {unix_group:4} {unix_mode:4} {unix_owner}")

We used the mlsd() method that calls FTP's MLSD command, it returns a tuple that contains the file name and the file meta data, we extracted everything and printed them to the screen. Notice I used TYPE I command to change the type of transfer into binary image, this is because size() will raise an exception if it's not the case.

We also used our previously defined function get_datetime_format() to convert the date returned by the FTP server into a more human readable format, here is a truncated output of the above recipe:

************************************************** MLSD **************************************************
File Name                      Last Modified       Size   Perm  Type GRP  MODE OWNER
backbone.t3-ps.Z               1991/07/30 11:28:13 23.39KB adfr  file 1    0644 1407
backbone.t1t3-ps.Z             1991/07/30 11:28:41 24.56KB adfr  file 1    0644 1407
ripe-map06-netnums.ps.Z        1991/07/08 09:57:23 29.54KB adfr  file 1    0644 1407
edlana4bw.ps.Z                 1992/06/17 13:30:40 63.34KB adfr  file 2005 0644 1407
...<SNIPPED>...

MLSD command is the current FTP standard of formatting directory listings, it was introduced on RFC 3659.

Finally, after working with the FTP server, it's time to quit and close the connection:

# quit and close the connection
ftp.quit()

Alright, that's it for the tutorial. As a conclusion, you shouldn't use LIST command (using dir() method in Python) now, MLSD is the way to go, even though some FTP servers still doesn't support MLSD, NLST command is still an alternative.

Related: How to Brute Force FTP Servers in Python.

Happy Coding ♥

View Full Code
Sharing is caring!



Read Also





Comment panel

   
Comment system is still in Beta, if you find any bug, please consider contacting us here.