Integrate a Sources content collection.
Plan: Introduce Sources as Accessible Content
Objective
Render the
content/sources markdown files through a dynamic render pipeline with radically flexible frontmatter handling, graceful error handling, and a tabbed index page for browsing by folder.Current State Analysis
Sources Directory Structure
The
sources/ content directory contains 247 markdown files organized in:- 11 top-level folders: Books, Brand Content, Events, Lectures, Media, Meetings, People, Reports, Source Extracts, UGC Communities
- 13 root-level files (e.g., OpenAlternative.md, CB Insights.md, Cursor Directory.md)
- Nested subdirectories (e.g., People/Influencers, Source Extracts/GitHub Repos)
Content Characteristics
Based on file inspection:
- Varied frontmatter: Some files have rich frontmatter (url, og_title, tags, etc.), others have minimal frontmatter (just date_created), some may have none
- Filenames as fallback titles: Many files use the filename as the primary title (e.g., "OpenAlternative.md" → "OpenAlternative").
- We should use the
titlefalling back toog_titlefalling back tofilename(there is an Astro specific word for this, which I can't remember)
Sensitivity of file system paths to urls in Astro SSG
- We've had trouble making sure the various backlinks work across different content that shows up with different site urls.
content/specs/Project-Routing-Fix-Complete-Implementation.mdis a good example of this.
- Backlinks present: Content may contain Obsidian-style
[[backlinks]] - Backlinks point to this file from other rendered content, so we need to review and set the path to this directory or page through:
- the
utils/routePaths.tsutil. - the
pages/apicode, which you might need to inspect.
Existing Patterns to Follow
- content.config.ts: Uses glob loaders with
resolveContentPath()and permissive schemas with.passthrough() - more-about/[...slug].astro: Uses
processEntries()from@utils/slugifyfor consistent slug generation - ReferenceLayout.astro: Tab-based navigation with counts and word counts
- VocabularyPreviewCard.astro: Simple card component for listing items
Implementation Plan
Phase 1: Content Collection Configuration
File:
src/content.config.tsAdd a new
sourcesCollection: typescript
const sourcesCollection = defineCollection({
loader: glob({
pattern: "**/*.md",
base: resolveContentPath("sources"),
generateId: ({ entry }) => {
// Preserve directory structure in ID for nested folder routing
return entry.replace(/\.md$/, '').toLowerCase();
}
}),
schema: z.object({
// Ultra-permissive schema - everything optional
title: z.string().optional(),
url: z.string().optional(),
date_created: z.union([z.string(), z.date()]).optional(),
date_modified: z.union([z.string(), z.date()]).optional(),
tags: z.union([z.string(), z.array(z.string())]).optional(),
publish: z.boolean().optional(),
}).passthrough() // Allow any additional frontmatter
}); Add to exports:
typescript
// In paths export
'sources': resolveContentPath('sources'),
// In collections export
'sources': sourcesCollection, Phase 2: Dynamic Route Handler
File:
src/pages/sources/[...slug].astro (new file)Key features:
- Graceful error handling in
getStaticPaths()- wrap entry processing in try/catch - Fallback titles from filename when frontmatter title is missing
- Consistent slug generation using existing
getReferenceSlug()utility - Error boundaries for individual page rendering
astro
---
import { getCollection } from 'astro:content';
import Layout from '@layouts/Layout.astro';
import OneArticle from '@layouts/OneArticle.astro';
import OneArticleOnPage from '@components/articles/OneArticleOnPage.astro';
import { getReferenceSlug, toProperCase } from '@utils/slugify';
export const prerender = true;
export async function getStaticPaths() {
const sourcesEntries = await getCollection('sources');
const paths = [];
const errors = [];
for (const entry of sourcesEntries) {
try {
// Generate slug from entry ID (preserves folder structure)
const slug = getReferenceSlug(entry.id);
// Derive title from filename if not in frontmatter
const filename = entry.id.split('/').pop()?.replace(/\.md$/, '') || entry.id;
const title = entry.data.title || toProperCase(filename);
paths.push({
params: { slug },
props: {
entry: {
...entry,
data: {
...entry.data,
title, // Ensure title is always present
}
},
folder: entry.id.includes('/') ? entry.id.split('/')[0] : 'root'
}
});
} catch (error) {
// Log error but don't fail the build
console.warn(`[SOURCES] Skipping ${entry.id}: ${error.message}`);
errors.push({ id: entry.id, error: error.message });
}
}
if (errors.length > 0) {
console.warn(`[SOURCES] ${errors.length} entries skipped due to errors`);
}
return paths;
}
interface Props {
entry: any;
folder: string;
}
const { entry, folder } = Astro.props;
// Build content data for components
const contentData = {
path: Astro.url.pathname,
id: entry.id,
title: entry.data.title,
contentType: 'sources',
folder
};
---
<Layout
title={entry.data.title}
frontmatter={entry.data}
>
<OneArticle
Component={OneArticleOnPage}
title={entry.data.title}
content={entry.body || ''}
markdownFile={entry.id}
data={contentData}
/>
</Layout> Phase 3: Index Page with Tabbed Navigation
File:
src/pages/sources/index.astro (new file)Design approach:
- Tab for each top-level folder + "All" tab
- Simple card list showing filename-derived titles
- Counts per folder displayed in tab badges
- Client-side filtering (similar to toolkit TagColumn pattern) OR static pages per folder
astro
---
import Layout from '@layouts/Layout.astro';
import { getCollection } from 'astro:content';
import { toProperCase, getReferenceSlug } from '@utils/slugify';
const sourcesEntries = await getCollection('sources');
// Group entries by top-level folder
const entriesByFolder = new Map<string, typeof sourcesEntries>();
entriesByFolder.set('root', []); // Files not in a subfolder
for (const entry of sourcesEntries) {
const parts = entry.id.split('/');
const folder = parts.length > 1 ? parts[0] : 'root';
if (!entriesByFolder.has(folder)) {
entriesByFolder.set(folder, []);
}
entriesByFolder.get(folder)!.push(entry);
}
// Sort folders alphabetically, but keep 'root' first or last as preferred
const folders = Array.from(entriesByFolder.keys()).sort((a, b) => {
if (a === 'root') return 1; // Put root at end
if (b === 'root') return -1;
return a.localeCompare(b);
});
// Process entries to ensure they have titles
const processedEntries = sourcesEntries.map(entry => {
const filename = entry.id.split('/').pop()?.replace(/\.md$/, '') || entry.id;
return {
...entry,
slug: getReferenceSlug(entry.id),
displayTitle: entry.data.title || toProperCase(filename),
folder: entry.id.includes('/') ? entry.id.split('/')[0] : 'root'
};
}).sort((a, b) => a.displayTitle.localeCompare(b.displayTitle));
---
<Layout title="Sources" description="Browse our collection of sources and references">
<div class="sources-container">
<h1>Sources</h1>
<p class="description">A collection of references, people, books, and other sources we've gathered.</p>
<!-- Tab Navigation -->
<div class="tab-nav">
<button class="tab-btn active" data-folder="all">
All <span class="count">{sourcesEntries.length}</span>
</button>
{folders.map(folder => (
<button class="tab-btn" data-folder={folder}>
{folder === 'root' ? 'Uncategorized' : toProperCase(folder)}
<span class="count">{entriesByFolder.get(folder)?.length || 0}</span>
</button>
))}
</div>
<!-- Content Grid -->
<div class="sources-grid">
{processedEntries.map(entry => (
<div class="source-card" data-folder={entry.folder}>
<a href={`/sources/${entry.slug}`} class="source-link">
<span class="source-title">{entry.displayTitle}</span>
{entry.folder !== 'root' && (
<span class="source-folder">{toProperCase(entry.folder)}</span>
)}
</a>
</div>
))}
</div>
</div>
</Layout>
<script>
// Client-side tab filtering
document.addEventListener('DOMContentLoaded', () => {
const tabs = document.querySelectorAll('.tab-btn');
const cards = document.querySelectorAll('.source-card');
tabs.forEach(tab => {
tab.addEventListener('click', () => {
const folder = tab.getAttribute('data-folder');
// Update active tab
tabs.forEach(t => t.classList.remove('active'));
tab.classList.add('active');
// Filter cards
cards.forEach(card => {
const cardFolder = card.getAttribute('data-folder');
if (folder === 'all' || cardFolder === folder) {
card.style.display = '';
} else {
card.style.display = 'none';
}
});
});
});
});
</script>
<style>
/* Styles following existing patterns from ReferenceLayout */
</style> Phase 4: Reusable Components (Optional Enhancement)
If the index page becomes complex, extract these components:
File:
src/components/sources/SourcesNavRow.astro- Tab buttons with folder names and counts
- Mirrors
ReferenceNavRow.astropattern - Uses an easy to understand json config to define what tabs to show and how to group the sources.
File:
src/components/sources/SourcePreviewCard.astro- Simple card showing title and optional folder badge
- Mirrors
VocabularyPreviewCard.astropattern
Phase 5: Route Configuration for Backlinks
Critical: The site uses a centralized route management system. For backlinks like
[[sources/OpenAlternative]] to resolve correctly, we must register the sources content path.File:
src/utils/routing/routeManager.tsAdd to
defaultRouteMappings array: typescript
const defaultRouteMappings: RouteMapping[] = [
// ... existing mappings ...
{
contentPath: 'sources',
routePath: 'sources'
},
// ... other mappings ...
]; File:
src/utils/routePaths.tsAdd to
ROUTE_PATHS constant: typescript
export const ROUTE_PATHS = {
// ... existing paths ...
SOURCES: {
BASE: '/sources',
},
// ... other paths ...
} as const; File:
src/utils/routing/routeManager.ts (optional enhancement)Add
sources to PRIORITY_CONTENT_PATHS if you want bare [[SomeSource]] backlinks to resolve: typescript
const PRIORITY_CONTENT_PATHS = [
'tooling/Portfolio',
'tooling',
'vocabulary',
'concepts',
'sources', // Add this for bare source name resolution
]; Why this matters:
- The
transformContentPathToRoute()function uses these mappings to convert[[sources/OpenAlternative]]to/sources/openalternative - Without this mapping, backlinks pointing to sources will resolve to
/404 - The route manager caches resolutions for performance
URL Normalization (already handled globally):
The backlink system already normalizes casing and spaces through this chain:
remark-backlinks.tsreceives[[sources/Open Alternative]]or[[Sources/OpenAlternative]]- Calls
transformContentPathToRoute(path)inrouteManager.ts routeManager.tsline 251 normalizes viagetReferenceSlug(input):- Splits path by
/ - Calls
slugify()on each segment (lowercases, converts spaces to hyphens) - Rejoins with
/
remark-backlinks.tslines 54-58 additionally slugifies each URL segment
Result:
[[sources/Open Alternative]] → sources/open-alternative → /sources/open-alternativeFiles involved in normalization:
src/utils/slugify.ts-slugify()andgetReferenceSlug()functionssrc/utils/routing/routeManager.ts-transformContentPathToRoute()at line 251src/utils/markdown/remark-backlinks.ts- additional slugification at lines 54-58src/utils/backlink-parser.ts- delegates totransformContentPathToRoute()
Verification needed: Test that backlinks like
[[sources/Brand Content/Some File]] correctly resolve to /sources/brand-content/some-file (handling both the space in "Brand Content" and any casing variations).Phase 6: Title Fallback Chain
Per your feedback, implement a title fallback chain:
typescript
// In getStaticPaths() and index page
const getDisplayTitle = (entry: any): string => {
// Priority: title → og_title → filename
if (entry.data.title && entry.data.title.trim()) {
return entry.data.title;
}
if (entry.data.og_title && entry.data.og_title.trim()) {
return entry.data.og_title;
}
// Fallback to filename with proper casing
const filename = entry.id.split('/').pop()?.replace(/\.md$/, '') || entry.id;
return toProperCase(filename);
}; This follows Astro's "data cascade" pattern where frontmatter properties cascade with fallbacks.
Phase 7: Error Handling Strategy
Build-time Error Handling
In
getStaticPaths(): typescript
// Wrap each entry in try/catch
try {
// Process entry
} catch (error) {
console.warn(`[SOURCES] Skipping ${entry.id}: ${error.message}`);
// Continue with other entries
} Render-time Error Handling
In the layout/component:
typescript
// Defensive content handling
const body = entry.body || '';
const title = entry.data?.title || toProperCase(filename); Collection Schema
Use
.passthrough() to accept any frontmatter structure without validation errors.File Checklist
| File | Action | Priority |
src/content.config.ts | Add sourcesCollection | P0 |
src/pages/sources/[...slug].astro | Create dynamic route | P0 |
src/pages/sources/index.astro | Create index with tabs | P0 |
src/utils/routing/routeManager.ts | Add sources route mapping | P0 |
src/utils/routePaths.ts | Add SOURCES route constant | P0 |
src/components/sources/SourcePreviewCard.astro | Create (optional) | P1 |
src/components/sources/SourcesNavRow.astro | Create (optional) | P1 |
Testing Strategy
- Build test: Run
pnpm buildand verify no critical errors - Dev server test: Navigate to
/sourcesand verify index loads - Route test: Click through several source entries from different folders
- Edge case test:
- File with no frontmatter
- File with minimal frontmatter
- Deeply nested file (e.g.,
People/Influencers/SomeInfluencer.md)
- Tab filtering test: Verify each folder tab filters correctly
- Backlink test:
- Create a test backlink
[[sources/OpenAlternative]]in another file - Verify it resolves to
/sources/openalternative(not/404) - Enable
DEBUG_BACKLINKS=truein.envto see resolution logs
- Title fallback test: Verify files display correctly:
- File with
titlefrontmatter → shows title - File with only
og_title→ shows og_title - File with no title fields → shows filename in proper case
Risks and Mitigations
| Risk | Mitigation |
| Large number of files (247) causing slow builds | Use static generation, consider pagination if needed |
| Inconsistent frontmatter causing render errors | Ultra-permissive schema + defensive coding |
| Spaces in folder names (e.g., "Brand Content") | Use getReferenceSlug() for URL-safe slugs |
| Missing content body | Default to empty string in render |
| Backlinks to sources not resolving | Add route mapping to routeManager.ts (Phase 5) |
| Existing backlinks from other content pointing to wrong URL | Verify route mapping matches content path exactly |
Future Enhancements
- Search functionality: Add search input like
SearchInput.astro - Word count display: Show content length like
more-aboutpages - Sort options: Allow sorting by date, title, folder
- Folder-specific pages: Static
/sources/books,/sources/peopleetc. - Related sources: Show backlinks between sources
- Google Books API : Use the Google Books API to get book information from the
urlfield in the frontmatter, especially the cover.
Estimated Scope
- Minimal implementation (P0 only): 5 files, ~250 lines of code
content.config.ts(additions)pages/sources/[...slug].astro(new)pages/sources/index.astro(new)utils/routing/routeManager.ts(additions)utils/routePaths.ts(additions)
- Full implementation (P0 + P1): 7 files, ~450 lines of code
- All P0 files plus:
components/sources/SourcePreviewCard.astro(new)components/sources/SourcesNavRow.astro(new)