[Question]: Preserving hyperlinks when converting PDF to markdown or json

[sarah114tran]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

Hi! I am currently trying to extract the citations from a series of PDFs. I need to convert the original PDF into either markdown or json, but I am noticing for both document versions the output does not preserve the hyperlinks. Please let me know if you have any suggestions!

This is my current code for parsing the PDFs:

all_markdown = []

for filename in os.listdir(folder_path):
    if not filename.lower().endswith(".pdf"):
        continue

    file_path = os.path.join(folder_path, filename)

    parser = LlamaParse(
        api_key=LLAMA_CLOUD_API_KEY,
        result_type="markdown",
        annotate_links=true         
    )

    try:
        docs = parser.load_data(file_path)
        md_text = docs[0].text       

        base_name = os.path.splitext(filename)[0]
        md_filename = base_name + ".md"
        md_path = os.path.join(output_folder, md_filename)

        with open(md_path, "w", encoding="utf-8") as f:
            f.write(md_text)

        print(f"Saved markdown: {md_path}")
        all_markdown.append({"filename": base_name, "markdown": md_text})

    except Exception as e:
        print(f"Error processing {file_path}: {e}")

[dosubot]

Hi @sarah114tran! I'm Dosu and I’m helping the llama_index team.

Hyperlink preservation when converting PDFs to markdown or JSON using LlamaParse is determined by the external llama_parse package, not by LlamaIndex itself. There’s no evidence in the LlamaIndex codebase or docs that setting annotate_links=true guarantees hyperlinks will be preserved in the output. If you need reliable hyperlink extraction, you may want to check the documentation or source code of the llama_parse package directly for details on how it handles links, or consider using alternative PDF parsing libraries like PyMuPDF, pdfplumber, or pdfminer.six, which allow explicit extraction of hyperlinks.

If hyperlinks are missing, a practical workaround is to extract them separately (using a library that supports link extraction) and merge them into your markdown or JSON output as a post-processing step. If you want help with code examples for alternative libraries or post-processing, let me know!

For more on how markdown output may lose formatting (including links), see this discussion: LlamaParse markdown output and formatting.
For details on the LlamaParse integration and its reliance on the external package, see the integration README.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[sarah114tran]

Hi @dosu -- please show me how to implement the practical workaround that you mention, and more specifically, code examples for alternative libraries or post-processing.

[dosubot]

Hyperlinks aren't preserved in markdown or JSON output with LlamaParse/LlamaIndex because the built-in readers don't extract them. You can work around this by using PyMuPDF (fitz), which lets you extract both text and hyperlinks from PDFs efficiently.

Here's a practical example using PyMuPDF to extract text and hyperlinks, then merge them into markdown or JSON:

import fitz  # PyMuPDF
import os

def extract_text_and_links(pdf_path):
    doc = fitz.open(pdf_path)
    all_pages = []
    for page_num in range(doc.page_count):
        page = doc.load_page(page_num)
        text = page.get_text()
        links = []
        for lnk in page.get_links():
            if lnk.get("uri"):
                # Get the rectangle and the text inside it
                rect = fitz.Rect(lnk["from"])
                link_text = page.get_textbox(rect).strip()
                links.append({"text": link_text, "uri": lnk["uri"]})
        all_pages.append({"page": page_num + 1, "text": text, "links": links})
    return all_pages

def merge_links_to_markdown(page_data):
    md = page_data["text"]
    for link in page_data["links"]:
        # Replace link text with markdown hyperlink if found
        if link["text"] and link["text"] in md:
            md = md.replace(link["text"], f"[{link['text']}]({link['uri']})")
    # Optionally, append a references section
    if page_data["links"]:
        md += "\n\n---\n**References:**\n"
        for link in page_data["links"]:
            md += f"- [{link['text']}]({link['uri']})\n"
    return md

# Example usage
folder_path = "your_pdf_folder"
output_folder = "your_output_folder"
for filename in os.listdir(folder_path):
    if filename.lower().endswith(".pdf"):
        file_path = os.path.join(folder_path, filename)
        pages = extract_text_and_links(file_path)
        for page in pages:
            md_text = merge_links_to_markdown(page)
            base_name = os.path.splitext(filename)[0]
            md_filename = f"{base_name}_page{page['page']}.md"
            md_path = os.path.join(output_folder, md_filename)
            with open(md_path, "w", encoding="utf-8") as f:
                f.write(md_text)

This approach gives you markdown files with hyperlinks preserved inline and/or as references. For JSON, you can serialize the pages list directly.

PyMuPDF is generally faster and more memory-efficient than other libraries for this task (source). If you want a different output format (e.g., all links at the end, or a custom JSON schema), let me know your preference!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}