-brown?logo=linux&style=for-the-badge){kind=icon}
-blue?logo=apple&style=for-the-badge){kind=icon}
- Lightnovel Crawler
This application uses Calibre to convert ebooks.
Install it from https://calibre-ebook.com/download
Without it, you will only get output in epub, text, and web formats.
For macOS, you need to manually add the path to Calibre.
Before starting lncrawl, use the command:
$ export PATH="$PATH:/Applications/calibre.app/Contents/MacOS"If you used a folder other than Applications during installation, replace /Applications/ with your path to Calibre.
⏬ Windows: lncrawl.exe
In Windows 8, 10 or later versions, it might say that
lncrawl.exeis not safe to download or execute. You should bypass/ignore this security check to execute this program.
⏬ Linux: lncrawl
It is recommended to install via pip if you are on Linux
⏬ MacOS: lncrawl
It is recommended to install via pip if you are on Mac
⏬ To get older versions visit the Releases page
📦 A python package named lightnovel-crawler is available at pypi.
Make sure you have installed Python v3.8 or higher and have pip enabled. Visit these links to install python with pip in Windows, Linux, and MacOS. Feel free to ask on the discussions page if you are stuck.
To install this app via pip:
$ pip install -U lightnovel-crawlerIn some cases you have to use python3 -m pip or pip3 or python -m pip.
Next, open your terminal and run:
$ lncrawl -h
If that does not work, you have not configured PATH variables properly. You can access it anyway using:
$ python3 -m lncrawl -h
The master branch contains the latest stable code. If you can not wait for it to be released in the PyPi, you can get it like this:
$ pip install -U git+https://github.com/dipu-bd/lightnovel-crawler.git#egg=lightnovel-crawler
The dev branch contains cutting-edge, sometimes unstable changes. To install it:
$ pip install -U https://github.com/dipu-bd/lightnovel-crawler/tarball/refs/heads/dev#egg=lightnovel-crawler
Docker is a convenient way to run it anywhere.
- Pull docker image:
$ docker pull ghcr.io/dipu-bd/lightnovel-crawler
$ docker tag ghcr.io/dipu-bd/lightnovel-crawler lncrawl
- Run using docker:
$ mkdir -p ~/.lncrawl
$ docker run -v ~/.lncrawl:/data --rm -it lncrawl
You can setup alias to the above command in your terminal's profile. e.g.:
alias docker-lncrawl=docker run -v ~/.lncrawl:/data --rm -it -p 8080:8080 lncrawl
- Run server with docker:
$ docker-lncrawl server
- First clone the repository:
$ git clone https://github.com/dipu-bd/lightnovel-crawler- Open command prompt inside of the project folder and install requirements:
$ pip install -r requirements.txt- Run the program (use python v3.8 or higher):
$ python lncrawl- Build docker from source:
$ docker build -t lncrawl -f Dockerfile .
- Run server using docker:
$ docker compose -f ./scripts/local-compose.yml up
$ lncrawl -h
Usage: lncrawl [OPTIONS] COMMAND [ARGS]...
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --verbose -l Log levels: -l = warn, -ll = info, -lll │
│ = debug │
│ --config -c PATH Config file │
│ --install-completion Install completion for the current │
│ shell. │
│ --show-completion Show completion for the current shell, │
│ to copy it or customize the │
│ installation. │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ version Show current version. │
│ sources Manage sources. │
│ crawl Crawl from novel page URL. │
│ search Search novels. │
│ server Run web server. │
│ discord Run Discord bot. │
│ telegram Run Telegram bot. │
╰──────────────────────────────────────────────────────────────────────────────╯
Run lncrawl server to start the server and visit http://localhost:8080.
$ lncrawl -ll server -p 8080
The default login for the web is:
username: admin
password: admin
For terminal usage, you can check help on each subcommands by -h:
$ lncrawl crawl -h
Usage: lncrawl crawl [OPTIONS] [URL]
Crawl from novel page URL.
╭─ Arguments ───────────────────────────────────────────────────────────────────────────────────────╮
│ url [URL] Novel details page URL. │
╰───────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────╮
│ --noin Disable interactive mode │
│ --all Download all chapters │
│ --first N [x>=1] Download first few chapters │
│ --last N [x>=1] Download latest few chapters │
│ --format -f [json|epub|txt|pdf|mobi|docx|rtf|fb2|a Output formats │
│ zw3|lit|lrf|pdb|rb|tcr] │
│ --help -h Show this message and exit. │
╰───────────────────────────────────────────────────────────────────────────────────────────────────╯
Visit the discussions page for more information. You can also post your query there too.
Run the following command to add a new source:
$ lncrawl sources create
It can auto generate the code using ChatGPT.
Request new one by creating a new issue.
We are supporting 460 sources and 389 crawlers.
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🔍 | https://arnovel.me/ | 23 |   |
| 🔍 | https://kolnovel.com/ | 1 | |
| https://rewayat.club/ | 10 | |
| Source URL | Version | Contributors | |
|---|---|---|---|
| https://cclawtranslations.home.blog/ | 71 |  |
|
| https://domentranslations.wordpress.com/ | 71 | |
|
| 🔍🔑 | https://novelasligeras.net/ | 3 |
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🔍🖼️ | https://anime-sama.fr/ | 4 |  |
| 🔍 | https://chireads.com/ | 10 | |
| 🔍 | https://lightnovelfr.com/ | 9 |  |
| 🤖🔍 | https://lnmtlfr.com/ | 9 | |
| 🤖🔍 | https://massnovel.fr/ | 1 | |
| https://noveldeglace.com/ | 3 | |
|
| https://xiaowaz.fr/ | 4 | |
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🔍 | https://blnovels.net/ | 6 | |
| 🔍 | https://centralnovel.com/ | 13 | |
| 🔍 | https://illusia.com.br/ | 1 | |
| 🔍 | https://phoenixnovels.com.br/ | 1 |
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🔍 | https://fenrirscans.com/ | 1 |
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🤖🔍 | https://docln.net/ | 11 | |
| 🤖🔍 | https://ln.hako.vn/ | 11 | |
| 🤖🔍 | https://truyenfull.vn/ | 5 | |
| 🤖🔍 | https://truyentr.info/ | 5 | |
- JSON
- EPUB
- TEXT
- WEB
- DOCX
- MOBI
- RTF
- TXT
- AZW3
- FB2
- LIT
- LRF
- OEB
- PDB
- RB
- SNB
- TCR
 |