Introduction

This tutorial is intended to organize a multi-phase GitHub project using submodules. It covers setting up individual repositories for different project parts (assets, backend, frontend) and linking them through a central repository for better management and clarity.

Requirements

To follow this tutorial, it’s best to have at least the following:

1. Git installed on your machine - this can be done following the installation for your operational system
2. Basic knowledge of how CLI works - I won't teach how CLI works (mainly because each Operational system has a different way of working when dealing with terminals/shells)
3. Basic knowledge of some Git/Github-specific terminology—I won’t explain any terms here. For any particular terms used that you don’t understand, check the appropriate documentation, as listed here, or anything about "git anatomy."
4. Basic knowledge of Git and GitHub

Note

Any CLI commands added to this document were created following Windows Powershell standards - small changes may be necessary on them if you are using a different terminal. Also, keep in mind that my code editor of choice is VsCode, so any screenshots displayed here are from this code editor - so a few things may be different on your UI, depending both on the code editor of your choice and the extensions installed!

---

Motivation

After a quick conversation about organizing a GitHub project, @mazal suggested creating this tutorial.

We discussed this on a community Discord Server and then moved to a screen share session, during which I gave @mazal a quick overview of how I organize my Git repositories.

---

Quick Overview

Below, I'll name a few things to help us understand the terminology I will use in this documentation.

1. Git - a source control system where you can track, revert, and commit changes to your folders/files
2. Repository - a folder which is being tracked by Git (this one can be created locally - your computer) or online
3. GitHub - one of the various online Git hosting providers where you can save (push) your repository’s files.

---

The Problem vs. The Solution

Problem

We developers often don’t organize our personal/organizational repositories. People organize things differently, so this tutorial is merely a suggestion based on my methods.

Example: We are working on a project that requires three different parts of development: assets acquisition and creation (images, references, design prototypes, and any other graphical resources created or collected as part of the "pre-production" stage); creation of a backend server/application (using any backend technology that we want); and creation of a frontend server/application (for example, a website that connects to the backend we created, also using any technology we want.)

Note

I won’t cover the process of creating everything mentioned above for this document. The idea is to provide an "optimal model" for working with projects requiring multiple phases/stages of development.

Assumption: In a typical scenario, it is logical to organize a project into the following folder structure:

Project
|─── Assets
|   |─── asset_file1.ext
|   |─── asset_file2.ext
|   └─── ...
|─── Backend
|   |─── backend_file1.ext
|   |─── backend_file2.ext
|   └─── ...
└─── Frontend
    |─── frontend_file1.ext
    |─── frontend_file2.ext
    └─── ...

Looking at the folder structure above, the question is: How do we organize this in a “nice way” using Git while hosting the entire project on our Github repo? The following is a possible solution I find most effective for these situations.

Proposed Solution | Breaking down the problem

To understand a bit about this so-called “nice way,” let’s keep in mind some essential concepts/names that I'll use in this document:

• Individual Repositories: those are the repositories that I create for storing the parts of the project and or other repositories which may or may not have any co-relation among them
• Main Projects Repositories - repositories created to "host" a collection of repositories which have some relationship between them
• Sub-projects / submodules - references created between the Main Project Repositories and the Individual repositories which are correlated to the same project

Important

The use of submodules for naming follows the official naming used by Git when working with a repository inside another - the main key of this method - using submodules to manage repositories.

With these concepts/namings in mind, the solution I present is represented in the diagram below:

flowchart LR
%%{init:{'flowchart':{'nodeSpacing': 80 }}}%%
subgraph local["Main Project Folder
on Local Machine"]
  direction LR
  al
  bl
  fl
end
subgraph al["Assets"]
  direction LR
  ald("asset_file1.ext
    asset_file2.ext
    ...
")
end
subgraph bl["Backend"]
  direction LR
  bld("backend_file1.ext
    backend_file2.ext
    ...
")
end
subgraph fl["Frontend"]
  direction LR
    fld("frontend_file1.ext
    fronted_file2.ext
    ...
")
end
subgraph repos["Repositories List"]
  direction BT
  po
  ao
  bo
  fo
end
subgraph ao["Assets Repository"]
  direction LR
  aod("asset_file1.ext
    asset_file2.ext
    README.md
    ...
")
end
subgraph bo["Backend Repository"]
  direction LR
  bod("backend_file1.ext
    backend_file2.ext
    README.md
    ...
")
end
subgraph fo["Frontend Repository"]
  direction LR
  fod("frontend_file1.ext
    fronted_file2.ext
    README.md
    ...
")
end
subgraph po["Project Repository"]
  direction LR
  pod["See deatils to the left"]
end
subgraph online["Github"]
  direction LR
  repos
  main
end
subgraph main["Project Repository Details"]
  subgraph asm["Assets Submodule"]
    ar("Last commited and pushed
        id of Assets Repository")
  end
  subgraph bsm["Backend Submodule"]
    br("Last commited and pushed
        id of Backend Repository")
    end
  subgraph fsm["Frontend Submodule"]
    fr("Last commited and pushed
        id of Fronted Repository")
    end
    .gitmodules
    README.md
end
al--"pushes to"---ao
bl--"pushes to"---bo
fl--"pushes to"---fo
ao--references---asm
bo--references---bsm
fo--references---fr
po--is detailed by-->main

Loading

Note

This diagram is a simple overview of the method I'll discuss in the next section.

Before proceeding, let’s break down the diagram by "linking" the items represented there with some of the terminology we'll require for this document.

1. We need an individual repository for each folder of our structure. Since we have the folders Assets, Backend and Frontend, we need an individual repository for them
2. Our root folder (the main project folder) also needs a repository for it - it's in this repository that we'll make the use of submodules to create the required links between the correlated repositories
3. We won’t push the actual repository for the details part of our diagram. Still, we will use a particular file created automatically by git—.gitmodules (more about this later) to make the links. This repository will store references to its “children repositories” just as the folders related to it are “children folders” for the project folder on our local machine.
4. "Pushes to" makes relation to the push (upload) process responsible for sending the files to GitHub
5. "References" indicates that the particular file mentioned above will contain a reference to the individual repository and incorporate that reference as a link to the actual last pushed commit of each one of the individual repositories

---

What to do next?

Proceed to the next section of this document, where I'll explain with as much detail as possible how to set up this structure on your Github and on your Local machine!