How to migrate WordPress to Scully

For quite a long time I was thinking about migrating my blog to JAMstack, ideally with a markdown support. I was looking for something like gatsbyjs or other alternatives. Most of them were fairly good, except the part that I would like to see my blog built with Angular in order to use all the nice features of the framework.

So, here comes the plan:

• Export posts to XML
• Convert XML to markdown
• Support for images
• Angular goodies
• Deployment

Export posts

The first thing you have to do is to export all your posts from WordPress. Luckily for us WordPress has a functionality to export all your posts into XML. You can find this option in your WP Admin navigation:

For more details check WordPress official documentation - Tools Export Screen.

After this operation you have your XML file, but it's still XML and you would like to have MarkDown format.

XML to MD

As a way to convert XML into MD within keeping blog format we can use wordpress-export-to-markdown tool:

Answer prompt questions carefully because it's better to keep the same structure of URLs.

Conversion finished? Well done! Now you have you md files with images. Doesn't Scully support images within md?

Blog images and Scully

Bad news for you. Scully does not know what to do with images, it skips them for conversion, but still it doesn't copy them in the right directory together with compiled HTML.

Good news - Scully has a plugin system. If you want to know how to write Scully plugins please check this article by Sam Vloeberghs, it's great!

Scully plugin to copy images

We want Scully to copy images from source of md files to compiled html files. For that to happen, we will create a small image plugin(image.scully.plugin.ts):‌

export function imageFilePlugin(raw: string, route: HandledRoute) {
  return new Promise((resolve) => {
    fs.copyFile(route.templateFile, './dist/static/images/' + route.data.sourceFile, (err) => resolve(''));
  });
}

There is yet neither ./dist/static directory, nor ./dist/static/images, so you need to create them before copying:

if (!fs.existsSync('./dist/static')) {
  fs.mkdirSync('./dist/static');
}
if (!fs.existsSync('./dist/static/images')) {
  fs.mkdirSync('./dist/static/images');
}

Now we need to register our plugin for all types of images that we want to support(you can add more if you need):

registerPlugin('fileHandler', 'png', imageFilePlugin);
registerPlugin('fileHandler', 'jpg', imageFilePlugin);
registerPlugin('fileHandler', 'gif', imageFilePlugin);

after some prettification, the final version of our image plugin (image.scully.plugin.ts) is:

import { registerPlugin, HandledRoute } from '@scullyio/scully';
import * as fs from 'fs';

if (!fs.existsSync('./dist/static')) {
  fs.mkdirSync('./dist/static');
}
if (!fs.existsSync('./dist/static/images')) {
  fs.mkdirSync('./dist/static/images');
}

export function imageFilePlugin(raw: string, route: HandledRoute) {
  return new Promise((resolve) => {
    const src = route.templateFile;
    const dest = './dist/static/images/' + route.data.sourceFile;
    fs.copyFile(src, dest, (err) => {
        if (err) {
          console.log(err);
        }
        console.log(`${route.templateFile} was copied to ${dest}`);
        resolve('');
      }
    );
  });
}

registerPlugin('fileHandler', 'png', imageFilePlugin);
registerPlugin('fileHandler', 'jpg', imageFilePlugin);
registerPlugin('fileHandler', 'gif', imageFilePlugin);

and now we need to include this plugin to Scully config (scully.blog.config.ts) to make it work:

import './src/image.scully.plugin';

export const config = {
  ...

preRenderer router option

After I created Image Plugin, Sander Elias (creator of Scully) recommended me to choose even simpler way - to use preRenderer router option:

export const config: ScullyConfig = {
  ...
  routes: {
    '/blog/:slug': {
      preRenderer: async (handledRoute: HandledRoute) => {
        ...
        return false;
      },
      ...
    },
  }
};

so we can just return false to let Scully know that we don't want to render this path. So we can put a condition:

const fileExtention = path.extname(handledRoute.data.sourceFile);
if (['.jpg', '.png', '.gif'].includes(fileExtention)) {
  return false;
}
return true;

and also we can add our copy functionality to the case when we have an image:

const src = path.resolve('./' + handledRoute.route + fileExtention);
const dest = path.resolve('./dist/static/images/' + handledRoute.data.sourceFile);
fs.copyFile(src, dest);

Important: Scully ignores images by default (but doesn't copy them yet), so to make it work and to get all the handledRoutes for images, you just need to register a 'dummy' image plugin, that will do nothing but letting Scully know that we're going to handle some extensions:‌

registerPlugin('fileHandler', 'png', async () => '');
registerPlugin('fileHandler', 'jpg', async () => '');
registerPlugin('fileHandler', 'gif', async () => '');

Parse tags from XML

It's useful to have tags from your posts as well. By default wordpress-export-to-markdown does not parse tags. I've created PR for it. Not sure how fast it's gonna be merged, so if you need tags you can use my forked version.

Double encoding

It looks like there is an issue with WordPress XML Export, so if you have many non-Latin symbols, for example, you are writing your posts in another language it will be encoded 2 times. Thus, when I did export (with wordpress-export-to-markdown), I changed this line to make it work also for non-Latin titles.

No tables and special symbols

Unfortunately wordpress-export-to-markdown doesn't recognize old good html tables, so if you had them in WP Posts be prepared to do it manually again in md.

Also if you used symbols like [, ], \, -, _, $ be prepared that they're gonna be ecranised with backslash to \[, \], \\, \-, \_, \$. Sometimes, especially in code blocks, it's not expected behaviour.

Angular Services: title, articles, tags, search

When you have all your information in place (in .md files), you could think about such a nice and obvious functionality for WordPress (as well as any blog) as page title, tags or search, and now you can do it all on the client side!

TitleService

Angular already has a title service, so you only need to inject this service

  constructor(
    ...
    private titleService: Title) {

and set a title based on your article:

this.scully.getCurrent().subscribe(article => {
  this.titleService.setTitle(article.title);
  this.article = article;
});

Article Service

Let's create our base service - Articles Service to manipulate with all the content. We will use scully.available$ stream for this, so:

getArticles(): Observable<Article[]> {
  return this.scully.available$;
}

but it's not that easy because if you have not only *.md files Scully will create an item for each file (yes, also for images), I opened an issue and hope it's gonna be resolved soon, but for now you need to filter only *.md files, so:

this.scully.available$.pipe(
      map((articles: Article[]) => articles.filter((article: Article) =>
        article.sourceFile?.split('.').pop() === 'md')));

for each article you have a date, so I would like it to have DESC order - new ones on top:

map((articles: Article[]) => {
  return articles.sort((articleA, articleB) => {
    return +new Date(articleB.date) - +new Date(articleA.date);
  });
})

it's also convenient to have a limit:

map(articles => articles.slice(0, limit))

here are we:‌

  getArticles(limit = 10): Observable<Article[]> {
    return this.scully.available$
      .pipe(
        tap(articles => console.log(articles)),
        map((articles: Article[]) => articles.filter((article: Article) =>
          article.sourceFile?.split('.').pop() === 'md')),
        map((articles: Article[]) => {
          return articles.sort((articleA, articleB) => {
            return +new Date(articleB.date) - +new Date(articleA.date);
          });
        }),
        map(articles => articles.slice(0, limit))
      );
  }

With the help of Article Service now you can output a preview list of your articles:

<app-article-preview [article]="article" *ngFor="let article of articles$|async"></app-article-preview>

Tags Service

Based on ArticleService we can get all the tags, also with a counter for each one that we can create a tag cloud after:

getTags(): Observable<Tag[]> {
  return this.articleService.getAllArticles().pipe(map(articles => {
    const tags = [];
    articles.forEach(article => {
      article.tags.split(',').forEach(articleTag => {
        const tag = tags.find(t => t.title === articleTag);
        if (!tag) tags.push({ title: articleTag, count: 0 });
        tag.count++;
      });
    });
    return tags;
  }));
}

Search

It would be sad if your blog doesn't not have an option to search (or to filter by tag). We already have ArticleService, so what we only need to do is to filter by tag:

articles.filter((article) => {
  if (!tag) {
    return true;
  }
  return article.tags.includes(tag);
});

or search query:

articles.filter((article) => {
  if (!searchTerm) {
    return true;
  }
  return article.title.includes(searchTerm) || article.tags.includes(searchTerm);
});

and now all together:

getFilteredArticles(tag: string, searchTerm: string, limit: number = 10): Observable<Article[]> {
  return this.getAllArticles().pipe(
    map( (articles: Article[]) => {
      return articles.filter((article) => {
        if (!tag) {
          return true;
        }
        else if (!article.tags) {
          return false;
        }
        return article.tags.includes(tag);
      });
    }),
    map(articles => articles.filter(article => {
      if (!searchTerm) {
        return true;
      }

      return article.title.includes(searchTerm) || article.tags.includes(searchTerm);
    })),
    map(articles => articles.slice(0, limit))
  );
}

Isn't it cool to have everything on frontend with the search that executes and shows result for less than a second?

Code Highlight

Btw, if you don't know you can also highlight your code blocks (i.e. <pre><code class="language-typescript"></code></pre>). For this you only need to activate this option in Scully config (scully.blog.config.ts):

setPluginConfig('md', { enableSyntaxHighlighting: true });

because by default it's switched off.

Deployment

You can deploy to any static hosting. It could be GitHub Pages, FireBase or Vercel. My personal preference is Netlify. We just need to add a deployment command:

ng build --prod && npm run scully

and setup distribution directory to ./dist/static

Partial compilation

If your blog has more than 100 posts you probably don't want to recompile all of them each time when you update one. For this you can use an option routeFilter and filter by only one section ( it's usually a year or a year and a month in WordPress):

ng build --prod && npm run scully -- --routeFilter "*2020/11*"

with such a flag Scully will regenerate only md files from 2020/11 directory.

You can go even further and use git to identify which files were changed in the last commit:

git show --name-only --oneline HEAD | tail -n +2 | grep 'blog/'

so the final command would be:

 npm run scully -- --routeFilter "$(git show --name-only --oneline HEAD | tail -n +2 | grep 'blog/' | xargs  | sed -e 's/ /, /g')" --scanRoutes

for more convenience you can put it into your package.json commands.

Conclusions

Scully team did a great job! Even taking into account all the small tweaks that you should do to make it work for your specific case. Scully is indeed a fair gatsby alternative for Angular. I did an experiment with my WordPress blog which made me confident to recommend it to you and, of course, to join Scully community.

All the helpful resources

• all the Scully blog code is here.
• And the site is life so you can play around - https://blog.stepansuvorov.com/
• Scully Docs
• "Custom plugins for Scully" by Sam Vloeberghs

...and, of course, you can ping me with all your questions and suggestions.